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AN OVEkVlEWOF THE OREGON STUDIES IN EDUCATIONAL RDD&E 



In tlie spring of 1970 the Training Branch of the U.S. 
Office of Education, National Center for Educational 
Research and Development, announced a plan to effect 
change in the preparation of edtcational RDD&E person- 
nel. Two factors led to the annow. cement. The underlying 
factor was the rather dramatic emergence in the past decade 
of development, diffusion, and evaluation activities as 
vehicles for educational improvement, and the attending 
need for qualified personnel to carry them out. The 
precipitating factor, however, was evidence that in spite of 
an investment of approxiinately 30 million dollars by the 
Federal Government to help training programs become 
more responsive to the. personnel needs created by these 
new activities, essentially the same number and kind of 
personnel were being prepared in 1970 as in 196S. 

The plan for change reflected a strategy that can best be 
described as “beginning at the beginning.” It incorporated 
three interrelated lines of activity: the creation of a 
conceptual and empirical base on which to build functional 
training programs; the design of more effective and efficient 
approaches to training; and the development of instruc- 
tional materi^s that reflect desir^ changes in both content 
and procedure. The propositions on which the plan rested 
were straightforward: (a) little was known about edu- 

cational development, diffusion and evaluation activities, or 
how they related to education^ research; (b) even less was 
known about the; training of personnel to carry out such 
activities; and (c) until both ; of these conditions were 
remedied the likelihood of desi^ng effective and efficient 
programs to prepare penmnnel'to <arry them out was slight. 
The plan as a whole was -coordinated so that the various 
activities within it would be (leteloped with sensitivity to 
each other, and so ' that they wc uld come together in 
completed fashion at apivpxiihately the same point in time. 

^(For additional detaus bn'the plan for change see Chapter I 
' in Volume I of the reportii^ the Oregon Studies.) 

The Ore^oiv> Studies,- carried out by the Teaching 
Research Division of. the Oregon- State System of Higher 

■ Education, were to contribute in a beginning way to the 
coricepiiial ahd empirical base, called for in the plan. As 
such they ySere to produce\fiye. products: a collection of 
detailed ,‘^case study” descriptions of projects that illus- 
trated' exemplary RDD&B activiHes within various edu- 
cational contexts; a rej^Ue, economically feasible method- 
ology by which to. collect die data needed to prepare the 
case studies; a conceptual; system or framewirk for viewing 
the domain of educational RDD&E that co jld be used as a 
guide to the classes of data' to ~be attended to in the case 
studies; cross-project apalysei- that highlighted the simi- 



larities and differences observed in the projects described, 
and that tested in rudimentary fashion the adequacy of the 
conceptual framework underlying those observations; and a 
compendium of the existing literature that pertained to 
either the nature of or the interactions between activities 
labeled educational research, development, diffusion and 
evaluation. These products are reported in five volumes: 



Volume 1. Summary Report (with Technical 
Appendices) 

Volume II. The Literature of Educational RDD&E 
Part One (Research, Evaluation, and 
Development) 

Part Two (Diffusion & Combinations of 
RDD&E) 

Volume III. Conceptual Frameworks for Viewing 
Educational RDD&E 



Volume IV. Profiles of Exemplary Projects in 
Educational RDD&E 
Part One (Research and Evaluation) 
Part Two (Development) 

Part Three (Diffusion) 

Volume V. A Methodology for the Study of 
Educational RDD&E 



Each volume in the series reporting the Studies has been 
designed to stand alone, but because each volume reports a 
different product, and each product can be understood 
fully only in relation to the other products, two “reader’s 
guides” to. the series have been prepared. The first involves 
brief summaries or abstracts of the contents of each of the 
five volum' s in the series. These appear on the inside of the 
back cover of the volume, and are intended to serve as a 
guide or overview to the series as a whole. A more Je mailed 
guide is provided by Volume I. In addition to serving as a 
general summary of the Studies, it contains descriptions of 
the developmental histories of the products reported in the 
various volumes, the relationships that exist between them, 
and the manner in which they have interacted over time. 
Accordingly, for the reader who wishes to determine 
quicl;!y what each of the five volumes in the series contains, 
turn to the inside of the back cover of the volume; for the 
reader wh.; wishes to understand how the volumes relate to 
one another, follow that by reading Volume I. 
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ABSTRACT 



This is the fifth in the series of volumes reporting the results 
of the Oregon Studies in educational research, development, diffusion 
and evaluation. The /olume's primary purpose Is to serve as a "how to" 
manual for individuals interested in replicating or expanding upon the 
data collected by the Studies. The two introductory sections of the 
volume, "Notes on..." and "Guide to...", discuss the development of the 
methodology and the organization of the methodology description in this 
volume. The remaining three chapters are devoted to a presentation of 
the methodology. The first of the three chapters describes the proce- 
dures involved in generating data from the sites selected for study. 

The second chapter describes data reduction activities, while the third 
chapter describes the procedures involved in the preparation of a pro- 
file description of the site visited. The epilogue following the three 
methodology chapters discusses briefly the nature of the methodology as 
a technological contribution. 



l‘RKKACl:: 



Tfie present volume contains a descr i.pt l(mi of the case studv mothod- 
olopy developed within the Oregon Studies. It dose r then the elements 
' within the methodology, the decision rules lo be lowed in applying 

* he methodology, and the steps to be taken in translating the data gen- 
' crated by the methodology into Its various reporting lorms. As such 

the volume is intended to serve as a training manual for persons wish- 
ing to apply the methodology in subsequent research efforts. 

The volume has been organiited into diapters that portray sequentially 
the steps taken in applying the methodology. The throe chapters that com- 
I)rise the volume, and the topic or "activity" headings within the first 
three chapters, reflect this sequencing principle. Thus, as the volume 
Is read, the reader will progress from the steps to bo taken in contact- 
ing a potential project for study through the various steps in data col- 
► lection, analysis and reduction, to the preparation of the case profiles 

and data displays that function as the repository for the data collected 
^ through the methodology. Category sets, decision rules, computer pro- 

grams for analysis, and detailed descriptions for use of the data files 
created in the Oregon Studies appear as appendices to the volume. So 
organized, the volume has utility as a general description of the method- 
ology as well as a manual to be used in preparing persons to use the 
methodology. 

By design, The volume has been prepared after the methodology had 
been developed, and after all data had been collected and processed in 
the form of case profiles, 'fhis was deemed essi^ntial, as early in the 
Oregon Studies the decision was made to let the methodology remain open 
to change throughout the life of the Studies. This was in keeping with 
the perception of the Oregon Studies as only the first of a series of 
empirical studies to be undertaken on the nature of educational RDD&E, 
a primary objective of the Studies being the development of a strong 
conceptual - methodological base for the subsequent studies. Obviously, 
data were also to be collected during the course of the Oregon Studies 
(see Volumes I and IV in the series of volumes reporting the Studies for 
a summary of the data collected), but they were to be collected as much 
I for the purpose of methodological development as the increased under- 

I standing of educational RDD&E that they would provide. The methodology 

! has undergone six major revisions (see NOTES ON THE DEVELOPMENT OF THE 

; METHODOLOGY) and has been tested for its applicability by being applied 

to the description of 20 ongoing educational RDD&E projects that varied 
! significantly one from another in size, setting, focus and content. 

Given the centrality of the methodological effort to the Oregon 
Studies, the requirement that it be fashioned against and consistent 
with the conceptual framework guiding the empirical thrust of the pro- 
ject, and that it be demonstrably effective (exhaustive, reliable and 
manageable) in the description of a broad range of ongoing RDD&E acti- 
vities, its development and testing became the most critical and the 
most demanding task within the Studies. It literally pervaded every 
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col 1 t'c t Lv(? 1 y , plc‘cec.“ on lius cffoil, insist tag LluU dcvt^lopmont 

oi nif t ti :)do logv rhji* crn:^ projcci TcHits. 

•Vbi tlie con' r ibut tons of l lu^sc aipc r in.'' the inef.hodoiogy [lavc 

f)e(;Mi groat, t.nc lacjoi burden of its J eve.l opnioro': l*,as been carried by t lie 
stall ol ih^ Oregon Studios. And, as Jijdicatpcij that burden was consi- 
dera’t.O.e, A rejativeiy unknown 'case ‘^-tudy" inethcdojogy had to be fashioned 
from v/hat was planned as a scandard tr«sk or job analysis methodol ogv ; a 
unit of analysis otliet than tasks or jobs had to be found; a format for 
tne pf'cseatat 2 on of ' case profiles" had to be estabi ished that could serve' 
both as a repository ^.or data and a readable description or "map" of a 
particular i^DDStl activity; a conceptual i'ramev/ork which "...differentiates 
and relates..." educational RDDjSiE had ic bo developed and then translated 
into a metnoaology that provided trustworthy data on all of the parameters 
Identified v/i thin, it; and both the conceptual framework guiding the meth- 
odology, and the laethodology icselip had to ne tested empirically for 
their e L f cc t i'^eness as descriptors of a wide range of RDD&E activities • 

All stall on the projecc were involved in one way or another w: th these 
various ta.sks, and in the Preface to Volume I in the series reporting the 

Studies are so recogniiied , The coniir ibu tiona of a number of staff to 

the methodology have been so greats however, rha" they must also be rec-' 
ognizfd here; Dr. Dale Hamreus as coordinator of methodoiogicai develop- 
ment in Phase I of che project ^ and with Dr. Harry Ammerman primary de^ 

velopers of t!ie quescionnaires; Dr, John WLllvamsou for suggesting out-^ 
put as a unit of .analysis; Mr. Gregory Thomas, Mr. Clark Smitli and Mr. 
boring ('arl as major contributors to the adoption and development of 
outputs as the central unit of analysis in the methodology ^ as opposed 
to jobs or tasks; Mr Clark Smith and Mr. boring Carl as the primary de- 
velopers of the on-.site interview methodology^ the on-site data reduction 
methodology, and category sets used for coding purposes; Mr. Gregory 
Thomas and Mr. Bill Hickok as the primary developers of the computer 



The names of persons contributing to the development of the methodology, 
as well as other aspect.s of the Studies. v?iil be found in the Preface 
to Volume I of the series of volumes reporting the Studies. 
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basrd data raanaf;ement system and cross-project analy.^t»s; Mr. I'arrel I 
Clukey as primary developer of the data tracking, systeni; and hr. Ammerman, 
Ms. Lee Green, Mr. Darrell Clukey, and Mr, Norman Crowhurst as primarv 
developers of the case profile format. Without the special al)illtli»s 
of these people, without their willingness to assist In the continuous 
Interchange between conceptual and methodological development, and with- 
out their willingness to work far beyond that which could be asked of 
them legitimately, the methodology would never have roaciied completion. 

My thanks, my sincerest expression of appreciation, and my con- 
gratulations to all for a job well done. 



H. Del Schalock 

Director of the Oregon Studies 



8 



V 



> 



(!(i 1 1 or ’ K I'r tM t* 



Mil- pr i'}M iM t imi (Uhl ju'intln^^ o tills voiunir tiLirks ilu* cornplctlnn 
ol t tPrli ;nul lln.il voluiiif* ot Mu’ jK*rh»s rf*j)orlin^» the Ore>;on 
Stiull**H, Krnin i lu* out net it. wau lnt(‘uoi*(i llun tlu* vnlutno bu ]>» op;u rd 
iiUit /iiul that it rcil'*rt any last minul<‘ revisions o( t;l.e Or(‘t;on 
Siudi(‘s m(*r hocif) 1 o)»y wliich mij^lu eiuer^ic* frori preparation of other 
vo)iim(»s, Tlius I h(» task was one of ilocunr r t ing , throughout tlie 1 IJ c 
of the Oregon Studies, the ’processes bein^^ eripioyoci, document I np, Liu* 
chain;es, and fip.Jiv assm iae, tie edited u'.’.curacy of I he fint'ivl version. 

Ih^yond the iiiattei of Lite content to he prest'nt(*d In tlie volume 
was the matter of the presentation ltS(*li . 'lie* question arose as to 
how one can present a description ol a relatively complex set of 
hehaviors in sutrh a way as to maintain souie sense oC the perspective 
requirc'd to und(*rstaru: the interrelationships ot the processes. 

Advisors /md consultants tn the Oregon Studies liad, at one tune or 
anotiier, observed dial the stall mij'h: consider ’’turning the metli- 
odologv on itself.” With that tnought in mind, the editors decided 
to adopt tlie strategy employed by the ruediodoiogy , i.e., to make 
exf)licit tlie various significant and essential elements of the meth- 
odology through die means of stating them in output form. It is 
hoped that witli clu* assistance of the section on Guide to Manual 
Organization and Use, the reader will linn l;is way through the meth- 
odology with understanding and meaning, 

The ed itors of this volume would be r (»miss in not extending 
a separate set of acknovledg inients to those related to its prepara- 
tion. First, ail past and present staff members of the Oregon 
Studies, in effect, ’’authored" the methodology as such. Various of 
them participated in maintaining its documentation through periods of 
time within the effort. These persons have been acknowledged in the' 
director’s prefac.^s contained in thia and other volumes reporting the 
Studies. Next, ui an acknow;ledgement of, and appreciation for the 
expression of faith demonstrated by H. Del Schalock, Oregon Studies 
Director, in permitting the editors free license in the design and 
preparation of the volume. To those advisors and consultants who 
prompted us to turn the methodology on ourselves, our thanks. The 
exercise contributed immeasurably to our own better understanding 
of what needed to be presented . And finally, to the remaining sec- 
retarial staff, both birth and death having taken its toll, our 
deepest appreciation for attention to levels of detail beyond the 
call of duty; Jan Mishler, Bea Lindahl, Penny Lane, Mary Hamann, 

Mary Abusharr, Anita Jones and Delphine Freshour. 



The Editors 
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Ten speclii Lrticn.s guided the development of the methodology em- 
pKyed in the Oreg'-n Studies. Four of the 10 came from the proposal; 
four from the external reviev^s to which the methodology was submitted; 
and two were added by project staff as the methodology evolved. The 
specifications stated in the proposai included: 

1. Tile methodology would p-roduce, in case study format, trust- 
worthy descriptions of the "competencies** needed by staff 
to carry out tlie operations involved in i<DD&E activities . 
Descriptions were to include (a) the indicators acceptable 
as evidenvic of such competencies; (b) tiie knowledges, skills, 
and sensitivities prerequisite to the d^.mons tration of compe- 
tencies ; and (c) the staffing patterns employed in the con- 
texts In wnich the competencies were studied; 

?• The methodology would be consistent with a *'preferred" 
conceptual framework; 

3. The methodology would be usable by people with widely 
varying backgrounds; and 

4. The mechcdology would be published in a form that would 
make it transportable. 

The specifications that emerged from the external reviews of the 
methodology included: 

5. The methodology would be open to modification throughout 
the project and be demonstrably applicable across a broad 
range of projects; 

6. The methodology would be open to the influence of what 
people in the field say about the nature of the domain and 
the significant variables within it; 

7. The methodology would be capable of collecting data on the 
’*dynaiTLlcs" of project operation; and 

8. Data collected by the methodology would be scored in 
retrievable form so that the data could be interrogated by 
others and could be added to by subsequent efforts. 

The specifications that were added by the Oregon Studies staff as the 
methodology evolved included: 

9. All labels used for purposes of classifying data would have 
intrinsic meaning; and 

10. The procedures used in onsite data collection would be 
minimally demanding of the time of the data collection 
team. 
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To enhance the possibility of achieving these specifications, 
certain quality iissurance procedures were employed t'nroughuiit the 
project. These included external reviews and internal cycling and 
recycling as the collected data themselves reshaped ciu*. methodology. 

To tile greatest extent possible, the statements of interviewees werj» 
used without significant change, reflecting the actual language of the 
domain. Data cl as s if i cat ions /categorizations were subjected to re- 
peated reliability tests to insure comparability of treaLiiient. 

Specifications and quality assurance procedures were not a mere 
set of rules ’*by which to play the game.” They wore mutually agreed 
upon guidelines which, it was hoped, would lead to a methodology 
which could be an outstanding tool for further exploration of this 
complex domain. Not only did the specifications establish basic para- 
meters for the methodology, they made possible the confidence on the 
part of the staff that the final form of the methodology would: 

(1) be a tool of continuing vclue in many contexts , (2) gather and 
display a picture of what actually goes on in the field, and (3) be 
usable by the staff or others in going far beyond the door that was 
opened with this initial project. 



Initially, the methodology was intended to evolve from a con- 
ceptual framework that was to be a synthesis arrived at by the combi- 
nation of four "conceptual papers" (see Vol. Ill of this series). 

That "preferred" conceptual framework was to serve as the conceptual 
base to the empirical as well as the methodological efforts within 
the Oregon Studies. The assumption underlying the plan was not only 
that a synthesis could be achieved, but that it should be — both for 
the benefit of the Oregon Studies and for the field as a whole. When 
tlie first drafts of the papers were reviewed, however, it was apparent 
that the notion of a synthesis would have to be abandoned. The views 
presented in the papers were so diverse, and were held with such con- 
viction, that the probability of reaching consensus on a single frame- 
work — especially in time for it to be of value to other aspects of 
the Studies — was near zero. As a consequence, it was decided in one 
of the review conferences (July 26-27, 1970) that only the paper to 
be prepared by the Oregon Studies staff (Schalock-Sell p;;per - see 
Vol. 3) was to guide and be reflected in the Oregon Studies methodology . 



The actual methodology development sought to operationalize the 
notions set forth by Schalc ck and Sell, and the final configuration of 
the methodology does follow this conceptual framework to a considerable 
degree. Specialized interpretations were necessary, on occasion, to 
provide operational definitions of concepts . In some instances these 
interpretations either dropped out or added to portions of the con- 
ceptual framework . In the main part , however, the final framework con- 
cepts for studying the domain of educational RDD&E activity correlate 
quite closely with the final data gathering and reporting procedures. 



The Oregon Studies were responsible for the develcpirent of a 
"case study" methodology by which relatively unspecialized personnel 
could generate and report data suitable for describing or mapping the 
people , operations and outputs of educational research , development , 
diffusion, and evaluation (RDD&E) activities. To enhance the power’ 
and meaning of the data supporting such mapping, the data were to be 
initially ordered and displayed in relation to the contexts from which 



'vi-re c o i U-M , ),o., thu project.s they uoscribud. To accomplish 
Lhi.s, n ”(Jase Prof jIo" fr^nnat wa^s designed for describing diverse 
proj<‘cLs, This f.. i ac.i J.i tated the ordering of project data to 
a.'ssurf iXMup rehens i.ve ness as well comparnb l 11 ty of data across proj- 
ects. The "Case i^rofile" became one of the two parallel development 
ei forts which evolved Lnt(.' the total Oregon Studies methodology. 

Wn i le the proposal and the conceptual framework established the 
basic j.irojccf parameters and the nature of the data to be sought, and 
the Case Prof lie t.ormat established the manner in which those data were 
to be reported, the second of the two-pronged development efforts dealt 
with data collect i.on procedures. Proceduraiiy . six phases in the 
development of the methodology can be identified. First, prior to ini- 
tiating data coilection activities, outline designs of both profile 
format and data collection strategies were presented for conference 
review (July 1970). Modifications of the designs were made in Phase 
l\r7o and data collection was initiated. The resulting data were pre- 
pared in three different profile styles for a single project. These 
were submitted for conference review, in conjunction with discussion 
of the dc^ta gathering procedures used, in October 1970. Following the 
October conference, the data gathering procedures were refined, a 
basic, profile format was decided upon, and further data collection was 
initiated. This third phase ended v?ith the presentation of the first 
four "full." profiles, and the data collection procedures used, to the 
third review conference (March 1971). In each of these three confer- 
ences, participants ineJuded the consultants to the Oregon Studies, 
trciining program designers, USOE personnel, and the conceptual paper 
wri ters . 

Following the three conference phases, three operational phases 
continued to shape the manner in which data were collected, classified, 
coded, ordered and presented. The fourth phase involved refinement of 
the methodology and preparation of six more profiles to reflect the 
essential content and structure of the more advanced profile design. 

The fifth phase involved an additional modification of the data 
coilection methodology and preparation of the final ten profiles to 
complete the twenty required of the Oregon Studies. Phase six involved 
the final review oi the twenty profiles, and finalization of the meth- 
odology as reported in this volume. 

As previously stated, the profile form and the data collection 
methodology evolved simultaneously throughout the project. The 
profile development , of course , guided the preparation of the twenty 
profiles and represents a significant aspect of the methodology. The 
methodology of profiling has been included as Chapter 3 as a "Guide to 
Profile Writers." The substance of this document is an integral part 
of the ove rail methodology reported in this volume . 

Though in retrospect the previously stated six developmental 
stages are identifiable, the evolution of the methodology was continu- 
ous throughout the life of the project. The following paragraphs pro- 
vide illustrative examples of the evolutionary process. 

Ihe arrival at a structured, open-ended interview technique is 
the first case in point. Several forms of data collection were 
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considered, including questionnaire. Job Inventory, and interview. To 
providti not only reasonably hard data but also to detect the flavor and 
dynamics of projects, Ihu onsite personal interview and questionnaire 
were chosen. However, the style of interview was arrived at only after 
several variations were tried. For example, an extremely structured 
approach was attempted in which a list of tasks was h.^nled to the 
Interviewee, and he was asked if he actually performed uaem. if the 
answer was affirmative, further probing ensued. This technique was 
felt to be too leading cind was therefore dropped. An cxtrenieiy open 
technique was attempted in whidi the interviewee was simply asked to 
tell, us about his work. This technique provided much conversation 
within which a few usable bits of data were contained. However, 
assuring any comparability of classes of data from one project to the 
next appeared to be ax'i almost ii surmountable problem. in addition, 
the extraction of data from the conversation was extremely time con- 
suming. This technique, too, wa ; deemed unacceptable. 

A structured approach that did not lead the interviewee was 
finally accepted as the most practical. It was structured in that it 
named the area in which an interviewee was to respond — a specific 
output. In relation to that output, certain kinds of daca were sought, 
but in an open manner . For example, in eliciting the standards an 
interviewee used for judging the acceptability of an output, the 
question might be, ’TLow do you know that your output is adequate?” 

This minimized the degree to which the question tended to suggest 
or lead respondents. (An unacceptable question would be, ”How do you 
measure the acceptability of your output?" — here, the use of a measure 
was suggested,) 

A second case in point relates to the conceptual-empirical 
derivation of category sets. The conceptual framework (Schalock-Sell) 
established the gross categories of data to be gathered as: the out- 

puts of work effort, the standards held for outputs, the operations 
required to produce specified outputs to specified standards, and the 
knowledges, skills, and sensitivities required to carry out those oper- 
ations (enablers). In attempting to describe the outputs of projects, 
and the standards, tasks and enablers that related to them, it was 
necessary to establish a number of category sets to handle the com- 
plexity that was found. Category sets were developed as an extension 
of the conceptual framework and their utility was tested by fitting 
collected data into meaningful categories. These initial conceptual 
categories were not sufficiently comp le te to contain all data obtained , 
The sets were therefore expanded on the basis of data collected. 

The collected data, and the expanded data sets were then used in 
modifying or more fully interpreting the conceptual framework. For 
example, the concept of "output" was initially expected co be used to 
categorize both "products" and "management responsibilities," However, 
the data collected supported expansion of the output categories to 
classify outputs as products , events , and conditions , 

The methodology utilized by the Oregon Studies was initially 
focused by the Schalock-Sell conceptual paper (Vol. Ilf), reviewed and 
modified over time in light of additional conceptual, development and 
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•■"Jiliiy (m‘ LiM.' ::ii.-Lli(.d')i(jgy Lo deal with iiicomiiip data (VoJ . IV). 
A', sufjli, tlii.s voKuiie represents a set of procedures wliidi have been 
sysLfin.ati ca 1 Jy refined in order to sharpen the ability to define 
hdtical ioiiai KDDW; . The appearance of new variations v/lJl require new 
re 1 i rieneiU s , Ihe next user of this set of procedures is encouraged 
to make additional modifications as they are seen to be necessary. 

Ihe end result of such modification cannot help but be a set of pro- 
cedures which are more capable of capturing that which is so familiar 
yet so unci ear to many of us. 



GLIDE TO MANUAL ORGANIZATION AND USE 



Five functions are served by providing the descriptive information 
in this volume. First , there is a documentation of the complete proce- 
dures that were developed. Second , others wishing to employ the same or 
similar procedures have available to them a full set of ^’how- to-do-it’' 
instructions, thus making the data portion of these studies replicable 
and open to expansion. Third , readers of the data-reporting volumes 
may acquire a better understanding of how the data were obtained, how 
to interpret such data, and how to understand their implications more 
effectively. Fourth , developers of training programs for RDD&E per- 
sonnel may acquire the knowledge necessary for making their programs 
more responsive to the needs of the field, and fifth , others wishing 
to interrogate or extend the existing data bank may acquire the skills 
necessary to do so. 

Characterizing this methodology in a familiar terminology is dif- 
ficult, sir.ee it seems to represent a means of data collection which 
is unique uithin educational RDD&E . It appears inappropriate to char- 
acterize it, in the terminology of either a job analysis approach, an 
anthropological approach, an hi?'. torical approach, an inventory approach, 
a questionnaire approach, or .'ir, interview approach, because in large 
part each of these approaches in itself was seen as too limiting for 
the task at hand. Following an txamination of each of these strategies, 
some aspects of each of them were utilized, but no one procedure was 
adopted carte blanche. 

The methodology is designed to provide a comprehensive descrip- 
tion of a project or organization at a particular point in time a 

"snapshot” of that organization. If used with several projects or 
organizations within a domain, a composite view of the domain can be 
obtained. Subsequent applications of the methodology with the same 
organi.zations can produce data which, when compared with original 
descriptions, may indicate trends. 'The data resulting from employment 
of the methodology fall within the following classes: 

1. Context 

2. People 

3. Outputs 

Context (general project descriptors) 

Since the context in which a project rests may account for con- 
siderable variance in what people do, it is necessary bo describe the 
context in which a project's work is performed. The following informa- 
tion is sought: 

1 . Obj ec tives of the pro j ec t 

2. Rationale for the project 

3. Project timelines 
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relates to people, it Is neocssary tu deiijicribe tlie project's personnel 
and the work acUi'nti.os they perform ni Ifolarion to their backgrounds 
and the roles they The fol Low I ng ,'hat a are obtained through ques 

Cionuaires : 



1. Backgrouno of traJuing and worjji experience people bring 
to Lheii' jOOo. 

2, Descriptions of jobs* the people hold. 

3» Support services and resources people require and use. 

4. Ppcceivotl requirements associated with jobs held. 

5. topfiasis given to various classes of work activities. 



()utputs ( outpu t and work requirement descriptors) 

In order to understand the nature of work within a project as it 
relates to the outcomes of work activity, the following sets of data 
are sought: 



1. Outputs generated by the project 

a. Pre ducts produced 

b. Events carried out 

c. Conditions established 

2. Work requirements associated with each output 

a. Standards by which the adequacy of an output or the 
processes generating it are judged 

b. Tasks (operations) performed in generating the 
output 

c. Enabling knowledges, skills, and sensitivities 
required to produce the output 

3. The relationships of an output to the other outputs of a 
project . 



The interactions of these data sets are displayed within Figure 1. 

The various combinations of these three prime classes of data are dis- 
cussed in Voliime I, Chapter 12 of the Oregon Studies. 

The organization of the methodology manual 

The three chapters of the methodology manual deal with the *'how 
to’* aspects of employing the methodology. The text of the manual 
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FIG. 1. Classes of information sought in describing a project, 
and their relationship to outputs of work effort. 
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’ ^yw i\ n h‘* . / i>; s<.;cp»? c\. Llu! chapuor? is i i. i us f rated in i'ii^uro 2. 

h..: i ; LU'i t r/atefc a c;liapcer. 
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f. Li 1 . M. ; ) ■ ■•.in-' «*Uu Lh‘: This lij;nar ;s in tended tc- 
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i I'c se-.’vnid /ir.^ca. .i*a is with data handling; acLiv'ties Reduction, 
sCoru.k»o, and prucessj.ng oi d-tua is couducteci by individuals skilled In 
reduction ni‘ data Jutu coding categories. Both the detailed descrip- 
tion Che eotapi.i'. e I i. *^ed u'ata files and the computer data preparation 
pTUH'.ed '.ives are d. c .nncnccrl in the apponaicas. 

'■du* tiiird ctiapi.t L teals with profiting a particular project, 

L ;i s cd Ml in le rina v i on ..in d d a t a ob tali le d for that p ro j e c t . Idie comp u- 
te.’!- is relied upon lO produce certain standard types of summary anal- 
yset; ^'oT th.c case pi.u'ile. The me t.houology is designed so that a 
.surniiuir i7,ed to mi of data present at ion represents approximately 50% of 
any c as e p r o f .i 1 1 -: . the o t i i e r ^07^ of the p ro .f i le Ls made up of s ub j e c - 
t i V c :i mp re s s i ovis based on direct ob s e r va 1 1 ims and tape recordings of 
Inter i evx\‘ > . 

Mgurc d j .1 I. us M.-tL 0 s the general flow of activity through the 
c:omplexi t ;.es of t'ne Oregon Studies Methodology. Each of the numbered 
boxes (activity b focks) in Eigure 2 corresponds to a similarly 
labeled ]-\eading rma associated text v/ithin the appropriate chapter. 

figure, t'nerafore. provides the reader with a comparative time- 
'i.ine chart for employing the loe thodoiogy . 

Tiu.s diagram dvocs not indicate the absolute time involved in 
.,u;i • vj ties nor does the size of any particular box Indicate its 
relati^^e importance. 'Ilie horizontal positioning of a box does indi- 
cate, however, the comparative beginning and ending points of an 
activity block, Tor exai^ple, note that Activity Block 6 (Question- 
naire Adnrlnis tratlon) and Block 7 (Onsite Data Reduction) occur, in 
part, during tine same time frame as Block 5 (Site Visit by a Data 
Collection Team) . Note further that Block 6 is finished prior to closing 
out a site visit and tiiau Block 7 continues and is to be completed by the 
time the site visit is closed out. 

Figure 2 illustrates the sequencing of major segments (activity 
blocks) of the muchodology primarily for the purpose of easy refer- 
ence Lo parts of the manual. Within each activity block, the outputs 
expected to emerge from that activity block have been identified. In 
tlie texts each activity block is introduced with an ^Output Index," 
a listing of those outputs. The identifying descriptors of each out- 
put consist of a letter and number, a descriptive title, and a code. 

The letter indicates chat the output is a product to be produced (P) , 
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cin event to be carried out: UO , ^ condition lO be ostabJLslied (C). 

ihe number is a soquentiai identity number for future reference (e.g., 
C-75). Following each descriptive title in the index is a two word 
code enclosed in parentheses. Tlie first word of the code describes 
the output as a contracted-for output (focal), a part ol a contracted- 
fur output (component), or an output which makes easier the production 
of a contracted-for output or a component output (facilitating). The 
second word of the code indicates the function it serves as either 
setting policy guidelines (policy), orchestrating the available re- 
sources (management), or fabricating the ends being sought (produc- 
tion). The complete output index listing for the first output, for 
example, is ”E-1 Selection of Site Contact Personnel (facilitating 
management) . " This should be read as follows : Selection of Site 

Contact Personnel is an event which is a component part of the method- 
ology and serves a management function. 

Following the Output Index in each activi ty block is a graphic 
illustration of how each output is related to others in that activi ty 
block. This illustration is called an "Output Map."- The Output Map, 
unlike Figure 2 , does not attempt to depict a process flow over time, 
instead it is an attempt to demonstrate the interdependencies of one 
output to another. It will be clear to the reader that a number of 
outputs which are produced in the time frame provided by Figure 2 are 
not interdependent until taken out of that time frame. 

The individual output maps can be combined to form an overall 
output map for Volume V (see Figure 3). The reader should be able 
to locate each activity block’s output map within this figure and 
begin to understand the interdependencies which do exist. 

When combined, the Output Indexes from each activity block provide 
an overall Output Index for the Oregon Studies Methodology as described 
herein. 

Output Index of Volume V . 

E-1 Selection of Site Contact Personnel (facilitating, manage- 
ment) 

E-2 Explaining Purpose of Desired Visitation (component, man- 
agement) 

E-3 Establishing Rapport ( component , management) 

E-4 Data Collection During Initial Contact (component, produc- 
tion) 



^ For a detailed description of output indexing see Chapter 1 
of this volume (Activi ty Block 2) . More complete definitions of the 
above terms are contained within the Glossary for this volume. 

^ For a detailed description of output mapping see Chapter 1 
of this volume (Activity Block 2) . 
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E'igure 3, Volume V Output Map. 
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:■ ,iii iiilLiai Site \lslL ( i , managuinen L ) 

•;-<> u* i\t ;\*. i VO ;;rot..‘iuent: to I’ar l Icipate as a bcUa Source (com- 
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iM‘ea .1 1 i a>; (or aa Initial Site Visit: ( iaci i i tatin^, man- 
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l'>b i)e.(i.3ifm :-us to Whether Site is an Appropriate Data Source 
( CO mp o . I (•. n i , uiai i a e n t ’) 

L-9 D: rec. t'jr ’ s Apjn‘ova.1 ior Data Coiicetion Team Visit- 

ation 0 o 1 np ^ . a e u t , man age me n 1. ) 

D-JO Duos L 1 onua i re Data Collection (component, uroduction) 

P-ll Initial Context Map (component, production) 

P - i 2 T n i . J a L 0 u L p u L 1 iid x c oinp ou an t , p ro d u c t ion ) 

P-J 1 Mil til'll DuCput Map (coniponont, production) 

Study of DnsLte Data Gathering Operations (facilitating, 
production) 

l>i3 Group Discussion of Onsite Data Grithering Operations 
iaci ii t at ing , production) 

D-J6 Demonstration of interview (facilitating, production) 

E-17 Critique cf Demonstration (faciiiuating, production) 
i>l8 PractlcG Interview (facilitating, production) 

Critique of i'ractice interview (facilitating, production) 

K-20 Practice ilecapoing (facilitating, production) 

E-21 Critique of Prt ctice Recapping (facilitating, production) 

C-22 SkilJfcd interviewers (component, production) 

E-23 Team Exercise (facilitating, production) 

E-24 Critique of Team Exercise (facilitating, production) 

C-25 Ability to Work as a Data Collection Team Member (facilitating, 
product ion) 

E-26 Composing a Data Collection Team (facilitating, management) 

E-27 Briefing a Data Collection Team (component, management) 

C-28 Data Collection Team Prepared for Site Visit (component, 
production) 

E-29 Team/Site Conference (component, production) 

P-30 Selected Outputs for Interviews (component, production) 

E-31 Interviews (component, production) 

?f-32 Deliveiing Questionnaires to Respondents (facilitating, 
management ) 

E-33 Explaining Questionnaires to Respondents (component, produc- 
tion) 

E“34 Collecting Questionnaires from Respondents (component, 
production) 

E~35 Recapping (component, production) 

E-36 Finalizing the Output Map (component, production) 

E~37 Contextual Debriefing (component, production) 

E“38 Comple ten^^ss Check (component, management) 

E-39 Final rJeeting (facilitating, management) 

E-40 Coding Q^*'=:s tionnaire Data (component, production) 

P-41 Coded Outputs (component, production) 

E-42 Confirmation of Data Collection Team’s Coding (facilitating, 
management ) 

E-43 Output Coding (component, production) 

E-44 Transfer of Output Codes to Recap Sheets (component, man- 
agement) 

K-45 Coding Standards (component, production) 

E-46 Coding Tasks (component, production) 
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E“47 Coding I-nablers vcoinponent, production) 

E-48 Empirioal Expansion of Catcpory Sets (componriit, production) 
E-49 Coding-“Consi s tency instrument Construction for the Data 
Collection Te im U'aciii tat ing , management) 

K-50 Coding-Consistency Instrument Construction f(*r the Coding 
Resolution Team ( facil: tat iiig, management:) 

E-5i Coding-Consistency Data Analysis (facilitating, management) 
F-52 Prototype Data Ba:"k (component, management) 

P-53 Data File Specifications (facilitating, management) 

E-54 Data File Creation (component, production) 

E-55 Verifying the Computer Inputs (component, management) 

E-56 Confirming the Logical Consistency of Items Coded to a Cate- 
gory (component, management) 

1^-57 Computer Program Descriptions (facilitating, management) 

E-58 Computer File Mcinipulation ( component , production) 

P-59 Control Card Descriptions (facilitating, management) 

P-60 Examples of Re trieval Requests (component , production) 

P-61 Computer Generated Profile Tables (component, production) 
C-62 Coordinated Handling of Data (component, management) 

P-63 Introductory Pages of Profile (component, production) 

P-64 Profile Chapter I: Overview (component, production) 

P-65 Profile Chapter 1 1 ; Parameters of the Proj ect (component , 
production) 

P-66 Profile Chapter III: Summary of the Data (component, 

production) 

P-67 Profile Chapter IV: Supplementary Data (component, produc- 

tion) 

P-68 Profile Chapter V: Project Dynamics (component, production) 

P-69 Profile Chapter VI: Implications for Training (component, 

production) 

P-70 Profile Appendices (component, production) 

P-71 Initial Case Profile (component, production) 

P-72 Edited Initial Case Profile (component, production) 

P-73 Reviewed Initial Case Profile (component, production) 

P-74 Final Case Profile (component, production) 

C-75 The Ability to Replicate Data with the Oregon Studies 
Methodology (focal, production) 



Resources required to effect the methodology 

Staffing to effect this methodology may be organized around the 
following functions : 

(^) Coordinators of major segments of activity. 

(y Interview teams , team leaders, and other institutional 
representatives who visit sites in the field. 

{c) Trainers of interviewers. 

(^) Data coders . 
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i(j) C<Miiputcr pr.u.^,ranimers and operators. 

{O waiters, 

i ' ro f i J e isdi to rs . 

(h) Hc^la analysts. 

The scope Chi' effort which utilises thlrj ir*e thoclology will dictate 
the number of people necessary for the acf. omp.l.ishment of the above 
functions. General qualifications of persons wlic are to perform these 
functions are rensouabl.y obvious from their titles: coordinators, 

interviewers, progt ai:ime rs , writers, coders, editors, etc. Familiarity 
with the general, nature of work performed by educational KDD&E proj- 
ects is a most desirable qualification for these persons. Case pro- 
file writers aiid data coders most particularly need an understanding 
of the total dojnain of R.DD<S<L, to place data within their appropriate 
contexts and tc understand the implications and interrelationships of 
such data. 

As a further aid in obtaining the complete implications and sig- 
nificance of a project, a case profile v;riter also serves as a member 
of the intervlev; team that visits the project site. Experience in 
the interviewing process is also highly advisable for both data- 
coilect. ion coordinators and data coders. Figure 4 illustrates one 
feasibLe organi 7,at tonal pattern of these job functions. 

The data coj le.cted within the Oregon Studies were summarized and 
placed upon, a computerized data file system. The data reduction, 
sorting, and retrieval programs are all written in FORTRAN IV computer 
language. The computer system required to employ this methodology may 
utilize either disc or tape capabilities. All operations programs, 
data file names, dimensions and contents are included in the volume, 

The system was designed to be maximally flexible because the 
specific directions to be taken in the Oregon Studies were not known 
at the time of the data system design, ITie user of the system will 
therefore find that any number of system modifications can be made in 
order to accelerate retrieval time for specific data items • 

Employment c.f thi.s methodology requires normally available cler- 
ical support with extensive typing and duplicating facilities. It 
further requires the use of portable battery-powered tape recorders. 
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FIG. 4. Organization of functional positions 

for data collection and data management. 
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Chapter I 



SITE VISITATION ACTIVITIES 



This chapter of the methodology volume is focused on data collec- 
tion processes . Its intent is to provide sufficient infc rmation to 
allow the reader to engage in collecting data from a preselected site. 
Included is detail on the procedural steps (a) during tl.e initial site 
contact, (^) in making the initial site visit, (c) in preparing a team 
of data collectors for the site (interview) visit, (^) in conducting a 
site (interview) visit, and (je) in closing out a site in terms of com- 
pleteness in data collection and creating a positive affect with the 
site staff. As can be seen from Figure 3, Cl\apter 1 is composed of 
Activity Blocks 1 through 8 and Block 18. While Activity Block 18 has 
influence over the first 8 blocks, it will be discussed in Chapter 2 
of this volume. The contents of this chapter then are Activity Blocks 
1 through 8. 



Activity Block 1 : Initial Contact of a Potential Si te . 



Output Index : 

E-1 Selection of Site Contact Personnel (facilitating, management) 
E-2 Explaining Purpose of Desired Visitation (component, manage- 
ment) 

E-3 Establishing Rapport (component, management) 

E-4 Data Collection During Initial Contact (component, produc- 
tion) 

C-5 Approval for an Initial Site Visit (component, management) 

C-6 Tentative Agreement to Participate as a Data Source (component 
management) 



The problem of site selection has been purposefully excluded from 
this volume, since it was felt that the problem of site selection must 
be addressed by the particular investigating agency chocking to use the 
methodology. The sample of sites selected for the Oregon Studies are 
described in Volume I along with the selection criteria and variables 
used in the selection process. 
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i:-l Select- ion o\' Six^_ Conv .a ut Parsonno j 

When a site has been identified as a potent. iai source of data, an 
Initial teiepiione contact is made with the site. This initial contact 
provides the I'irst opportunity for interaction between the investigating 
agency and the potential site. 

The agency representative making this contact must be a person who 
is not jn.lv knowledgeable about all phases of the methodology but also 
one '.viifj ;:an speak ^rnake binding agreements) for the investigating agency. 

In making tne initial site contact, any of the following personnel 
can u!:Llized (see figure 4); 

c’ t udy Director 

Coordinator for Site Selection, Visitation, and Scheduling 

Coordinator for Data Collection 

The most desirable choice, from a continuity viewpoint, is the Coordi- 
nator for Data Collection. 

1> 2 Explaining Purpose of Desired Visitation , and 
1 -j-Ji Es t a blish iug R apport 

Prior to making the telephone contact, it is helpful for the 
i.adlvlduai to become familiar with the background of the prospective 
site and/or project. Any number of information sources can be r.sed to 
obtain such backgi’ound knowledge. A project proposal can be one 
important source of data. It can provide preliminary information about 
the number of personnel involved in a project, its duration, the size 
and scope of proposed activities, the name and telephone number of an 
individual who can be contacted, an indication of the scope of work to be 
completed, and an indication of the focus for the overall project, i.e., 
research, development, diffusion, and/or evaluation. Promotional brochur* 
and articles written by staff members may also provide substantive infor- 
mation regarding the work of particular projects. 

During initial contact it is helpful to be familiar with the proj- 
ect i.n question in order to facilitate communication with the individual 
contacted. A good deal of information can be exchanged in the initial 
contact. Key elements needing immediate discussion include (^) the pur- 
pose of data collection; (M why the site was contacted; (jc) how the 
data are to be reported; and (^) the constraints which will be placed 



the site if they agree to participate. 
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It: is essential fur tlie site to understand tlie constraints placed 
upon tlie staff members cliosen for interview. The most imoortant of 
tliesu constraints is tlie amount of time needed for interviewing, since 
each interviewee will be expected to contribute a minimam of 2 hours 
fur the purpose of data coJ lection. ciccasionaJ ly this minimum of 2 
hours may need to be expanded to as mucli as 4 liours (wi r.h site con- 
currence), but this is tlie exception and not the rule. ior purposes 
of scheduling, the prospective site sliould plan on liaving the inter- 
view team on site for approximately 1 week. Not every site contacted 
may have either tlie time or tlie inclination to allow such an extensive 
amount of staff time to be utilized. This is particularly true in 
those projects whicli are in their initial phases or are rushing to 
finalize some set of activities. 

b-4 Data Collection During Initial Contact 
2 

Form 01 is designed to expedite the contact process. This form 
provides a permanent record of such critical data elemer.ts as the 
proj ect director's name , the contact individual , sub contractors who 
also may have to be contacted, relevant addresses, telephone numbers, 
etc. Though primarily used for initial contact purposes, the last 
attaclied page on Form 01 should be completed every time a contact is 
made with a site, lliis procedure enables an individual who is about to 
make a site contact to review the records of prior contacts to insure 
against duplicating business which has already been discussed with per- 
sonnel on the si te . Since any data collection effort of the size and 
scope described within this volume places a number of corstraints on a 
site to be visited, it is critical that the time spent xn talking with 
personnel on the site be as free of unnecessary redundc^ncy as careful 
planning can accomplish. 

Access to Form 02 during the initial telephone conversation has 
proved to be an expedient to the data collection process. Many of the 
information elements contained in Form 02 can be naturally and unob- 
trusively brought into a line of conversation. If written background 
data have been accurate and at all complete. Form 02 can be partially 
completed prior to the telephone contact and verified during the ini- 
tial conversation, 

The initial phone contact also provides an opportunity to clarify 
the focus of the project to be visited. 

C-5 Approval for an Initial Site Visit , and 

C-6 Tentative Agreement to Participate as a Data Source 

If the proposed site is willing and appears to be a viable source 
of data, the steps outlined above should result in obtaining a verbal 
agreement to participate as a source of data and es tab lishing a firm 
date for an initial site visit . 



All forms and workslieets referenced in this volume are located 
in Appendix 1* 
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Initial Visit to a Potential Site. 



Aellvitv 1'. 2: 



Out^)UL index : 



I>7 Preparia^.! I'or an Initial Site Visit (facilitating, management) 
P-8 Decision as to Wliether Site Is an Appropriate Data Source 
( componcm: , management) 

C--9 Project. Director's Approval for Data Collection Team 
Visitation (component, management) 

I>10 Questionnaire Data Collection (component, production) 

P-'ll initial Context Map (component, production) 

F'"12 Initial Output index (component, production) 

?-13 Initial Output Map (component, production) 



Output Map : 







E-7 



E-10 I i P-11 I I P-l~3 I 
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P-7 Preparing for on Initial Site Visit 

It is advisable, though not essential, for the institutional rep-- 
resentative who maae the initial telephone contact to be included as one 
oi the two staff members to initially visit a potential site. The other 
starf member should be the person who will assume leadership responsi- 
bilities for the data collection team which will return to the site for 
the major data collecting process.^ (The entire data collection team 
need not be identified at this time.) 

All travel arrangements required throughout the employment of the 
methodology should be made by one individual (essentially a secretarial 
responsibility). A master schedule board should be readily available 
for review by all project personnel. Such a master schedule can be or- 
ganized using "time in days" and "site name" as the axis dimensions. 
IVhile keeping such a table up to date consumes time, it provides a 



lliis arrangement of staff allows the minimum number of exchanges 
of data, both summarized and impressionistic. During the course of the 
Oregon Studies this combination of people was often but not exclusively 
utilized. 

tf'V 
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degree of closure for the individual interviewer making repeated trips 
to several sites. Following the scheduling of an initial site visit, 
the Coordinator of Data Inputs should be charged with adding to the 
master time schedule the date for the proposed visit, data collection 
team schedules, and personnel involved. 

Providing materials needed for the institutional representatives * 
onsite visitation is the responsibility of the Coordinator of Data 
Inputs. These materials might include: 

• Form 01 (Project Contact Record), started by the site- 
selection team. 

• Form 02 (Project Questionnaire). 

0 Forms 03 (Job/Task Inventory) and 04 (General Position 
Activities Questionnaire). Forms 03 and 04 should be 
available for each project staff member wh.j is to be 
contacted later by the interview team. (It may be de- 
cided to provide the forms prior to the team*s visita- 
tion, particularly when all staff can be readily iden- 
tified, e.g. , when the project contains only 3 or 4 
personnel. ) 

• Forms 05-A and 05-B (Output Recap: Index and Coding). 

0 Copies of any materials collected ab out the proj ect to 
date . 

0 Worksheets 08 (Dissemination Brochure), 11 (Contextual 
Cue Items), and 17 (Checklist of Project Data Inputs). 

In some instances, initial site visits will be conducted when 
only one individual from the proposed site is available. It is 
preferable for some or all of the staff of the proposed site to be 
involved during the initial site visit. This provides an oppor- 
tunity for rapport to be extended beyond the individual holding 
administrative responsibilities for the site in question. 

If the information collected on Form 01, Form 02, proposals, 
dissemination brochures, etc. has been thoroughly read and under- 
stood by the team riiembers, a common framework from which conversa- 
tion can be built is provided and the probability of »iS tablishing 
rapport is greatly enhanced. 

It is essential that the individuals making the ini tial site 
visitation have in their possession complete and up-to-date infor- 
mation describing the methodological limitations and constraints 
to be placed upon the proposed site. (ITiis presented problems for 
the Oregon Studies because of methodological changes which contin- 
ued to be made during the duration of the project.) This is par- 
ticularly true in terms of the amount of tiine the site must contrib- 
ute for purposes of interviewing. 
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).-b 1 m.-( lu WiifcLl uir Site Is an Appr opriate iJiata Source 

iL I.-. puSM for the initial sir -visitation Learn to make a 
jiKlj;i;iL'uL the basis of the initial site visit as to whether or not 
a visit i^y the data collection team is appropriate. I1ie degree of 
appropriateness may be ns.sessed in terms of (a) the cooperative 
iiat;u*e of the .staif. (b) the availability of the staff, (^) the 
possibility of schedvtllng a site visitation w;i. thin a reasonable 
length or' time, end (d) a subjective judgment on the extent and 
nature of the data lo be obtained. (A few sites were eliminated 
from the Oreg(.')n Studies sample at this stage of contact, primarily 
due to scheduling difficulties.) 

C-9 Projec t Director's Approval for Data Collection Team Visitation 

Assuming tliat the site visitation team believes the individual 
site represents a notentiai source of data, permission must be for- 
mally obtained from the project director, or an individual having 
authority to make such a binding commitment. 

h -10 St i onnai r a Data Collect i on 

If the decision to follow through with the site is made, the 
Initial site visit provides an opportunity to collect several classes 
of critical data. The completion and confirmation of Form 02 at this 
time is essential as it provides information needed for the data col- 
lection team visitation. This form should be completed during the 
initial site visit and collected by the visitation team. 

Form 03 (Job/Task Inventory) and Form 04 (General Position 
Activities Questionnaire) are instruments relating to individuals. 

These forms may be completed during the initial site visit, distrib- 
uted at that time and collected when the data collection team arrives 
on site, or administered and collected by the data collection team. 

A minimum requirement during the initial site visit is to obtain per- 
mission to administer these two forms. 

P- 11 Initial Context Map 

In addition to the collection of questionnaire data (Forms 02, 03 
and 04), an essential part of the site visit by the institutional 
representatives is the collection of information for the initial ver- 
sions of thL; Context Map, Output Index and Output Map. (These latter 
two outputs will be discussed separately.) The concept of a contex- 
tual map is used to provide further insights into additional classes 
and sources of data. The contextual map (see Figure 7) is designed 
to illustrate the environment in which the particular site resides. 

Several revisions of the contextual map usually need to be pro- 
duced before a satisfactory version is completed. Figures 5, 6, and 
7 present a sequence of map production. 

Figures 5, 6, and 7 demonstrate that the environment internal to 
the organization, the environment in terms of linkages with subcontrac- 
tors, and the environment in terms of to funding source should 

all be represented in a contextual map. 

6 * 







FIG. 5. An initial version of a contextual map. 





Sponsor 

(Office of Education) 



Cooperating Institutions 



Indiana 
Univer&j ty 



University 
of Illinois 
(Contractor) 



Purdue 

University 



English Department 



Projec^ 



Iri-University Directorate 
University of Illinois 
Indiana University 
Purdue University 


English 

Departments 


3 English Departments 
1 Education Dept. (Purdue) 





Consultants 

16 English Discipline 
(College and Univ.) 

9 School 

(8 Public, 1 Paroch.) 
2 Major consultants 
5 Readers 




FIG. 6. The first revision of a contextual map. 
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FIG. 7. Final version of a contextual map. 
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llic i.j 1 a framework a^>ninsL wludi Lhe da La 

' i ri ( i 1 .11 .an rcilnrt respoa/Jes obtained I'roni project staff 

rs . '!any Ls durin^^, an interview can be bett er understood 

wlieii sniiif ntide rs i and I up, of idie environment or context within which the 
[)roject liaa Lu funrtion is available to ail team merabers. Further, 
such ;i re[- re.seur.ai i on siiouJd also u ’.iniateily help the reader of the 
•:asc profile preoaro.d for each site, 'llie initial site visitation pro- 
vides an idt. at time for the first approximation to such a contextual 
map to be p,e ve r a e d . 

Tlie outputs from the initial site visitation can vary, dependent 
upon a number o] factors. lu the optimum circumstance, these outputs 
would incLude a completed Fora Ul, a completed Korin 02, permission to 
administer Forms 03 and 0^, an initial output index, an initial output 
map, and <in initial contextual map. In addition, firmly established 
interview dates, recommendations as to accommodations, and recommen- 
dations as to special conslderarions , including mode of transportation 
to and frem the site, should be set. 

A constraint which may affect the above output significantly is the 
amount of time the particular site is able to devote to the initial 
site visit. To accomplish all of the above on a medium size site, 
approximately 4 hours of directed conversation is necessary. If 
such time is not a^^aiiable or the complexity of the site does not 
permit , al ternative procedures may be necessary , 

P-1 2 Initial Output Index 

The outputs from ai'iy particular site are organized into an output 
index. 

Definitions : 

^ Output “ an identifiable outcome of targeted work activities 
.:ontributing to the realization of project goals, 

^ Output index - the outputs that are associated with a 

particular project lis ted according to 
the structure of output they represent. 

The terms product , event , and condition represent three identifi- 
able structures of outputs. These terms are discussed in a later part of 
this section. 

Def ini tions ; 

^ Product - a tangible or "hard” outcotae of work effort that is 
transportable at a given point in time, 

0 Event - an outcome of work effort that results in the occur- 
rence of an observable transaction or set of behaviors, 

^ Condition - an outcome of work effort that creates a desired 

circumstance expected to endure within the life of 
a project, or as a result of it. 
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Lf sufficient docuinents are available prior Lo tae initial site 
visit, a tentative listing of outputs can be constructed. Such listings 
focur. upon specific contracted~f or products that are typically cited in 
a project proposal. Distinction of output classes is usualJy not pos- 
sible at this stage. The following is an actual oui.put index which was 
developed prior to one of the Oregon Studies visitations. 

01. Statistical description of the data. 

02. Description of the sample. 

03. Description of the properties of variables. 

04. Computer programs. 

05 . Growth figures and patterns . 

06. Model of growth. 

07. Graphic illustrations of cause/effect among variables. 

08. Matrix of postulated effects. 

09. Simulation model. 

10. Field case studies. 

11 . ? relimi nary i n forma t ion . 

12. C'-mparison with simulation model. 

1 3. Observational data . 

14 . Empirical information . 

15. Technical papers. (5 in number) 

16. Monographs. (3 in number) 

In operational terms, the foregoing list of outputs is the result of 
intensive review of the available materials from a potential project site. 
Procedurally a particular site is characterized in terms of its outputs, 
based upon documents which were available prior to the site visitation. 
During the initial visit this list must be confirmed by the members of 
the site, with additions and deletions being made at their recommendation. 
Particular attention must be devoted to the identification of significant 
events and conditions, these being outcomes of project effort that are not 
typically identifiable from a reading of the project proposal. 

There are several points to keep in mind in the construction of 
the output index which have come to light from experience. Among them 
are : 

1. If descriptive information is available prior to the time of 
the initial site visit, some insights as to the outputs of 
the particular site may be built into a first approximation 
of the initial output index. This initial ”at-home’* attempt 
at identifying outputs can, in many instances, sa/e a con- 
siderable amount of site and project staff time. On the 
other hand, the printed material available from a site may 
provide little, if any, information necessary for the deter- 
mination of outputs . 

2. During the initial site visit it is helpful for the institu- 
tional representatives to have a portable tape recorder 
available so that the discussion of project outputs can be 
captured for later review and refinement. Documents which 
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lo.jy (in- iniorvlewer focua l»ia quc»st I on I nr. Inrlucic proj- 

('< i L Lmc‘ linos and statements of objoclivos. !f the Lutor- 
viewer tolLows the logical flow of time througl) a project, 
he must bn alert to avoid missing outputs wliid) often do not 
appear in such time orientations. The development of *’staff 
morale” or ’’parental coope ration” are examples of two out- 
put: conditions which might not normally be represented in a 
time -oriented chart . 

3. 1 1 is iuilpful (and will save a considerable amount of time 
later) fc.r the project staff to be available during the 

de termlrat ion of outputs. Frequently the project staff mem- 
bers pv(s able to make explicit what additional types of out- 
puts there are, and to clarify situations and circumstances 
which have occurred throughout the project. The initial 
visritation team must use their best judgment as to how much 
of the site’s time should be used for explicit and detailed 
clarincation of outputs. While sufficient data must be 
obtained to determi.ne the number of personnel it will take to 
interview a representative sample of the site, it is not 
necessary that the list of outputs be exliaustive of those 
produced by a project at this point in time. 

4. in organizing outputs into an output index, it will often be 
found that the list of products is fairly complete, while 
the list of events and conditions remain fairly sparse. 

This will be true particularly in situations where a time- 
line type of approach at identifying outputs has been used. 

In organizing the output index, the names of project staff 
members having a principal role in the realization of the 
output should be identified and associated with each of the 
outputs in the index. In many situations, more than one 
staff member will be responsible for or have knowledge about 
the same output. 

P-13 Initial Output Map 

Based upon the output Index, which is reviewed and modified as 
need be on the site, an initial attempt is made at developing an out- 
put map. 

Definition: 

0 Output map - a schematic ordering of project outputs in 

accordance with their interdependent relation- 
ships . 

In some instances it will be possible prior to the time of site visit- 
ation to construct an output map as well as an output index. In other 
instances the documentation will not allow the interdependencies to be 
identified. However, while on site, an initial output map must be 
developed and verified by the staff participating in the initial site 
visit. Without this map it will not be possible to assign interviewers 
to staff, identify the time it will take to visit a site, identify 
those outputs to be interviewed about, and come to understand the 
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intricacies of the data which will be fortiicoming . ‘Hie output map is 
not an attempt at displaying time-related dependencies. its purpose 
is the display of outputs and their interdependencies. The factor of 
time is only incidental in this map. It does not attempt to display 
an optimal sequence of completion of various outputs. It does display 
the effects or influences of outputs upon one another. The desired 
effect of reading an output map is a "picture” of the project being 
discussed in terms of the dependency relationships among the outputs 
the project seeks to achieve. 

Guidelines for the development of an output map 

The relationships between outputs are so diverse that only general 
guidelines can be stated concerning them. As aids to the reader a num- 
ber of figures follow to indicate sequential modifications which may 

need to be made in the construction of an output map. Figure 8 is a 
representation in pictorial form (i.e., output map) of the output index 
previously presented. "Hie map is reviewed by site personnel and modi- 
fied to their specifications . The modifications involved in the con- 
struction of an output map are represented in the sequence of Figures 
9, 10, and 11. Figure 9 is a map developed on the basis of documenta- 
tion available prior to the initial site visit. Figure 10 is the same 
map modified on the basis of data at the end of the initial site visit. 

Figure 11 is a map resulting from review and modification of the cross 

section of individuals interviewed on site, and is in the form that 
would subsequently appear in the case profile written for that site 
(see Chapter 3) . 

The following points will provide some additional direction in 
developing an output map; 

1. The output map does not intend to order time in relation to 
the outputs. As such, it is not a short-cut type of PERT 
chart. Time is incidental to the purpose of the output map. 
The dependencies of one output to another represent the 
critical aspects which the map attempts to demonstrate. A 
PERT chart, on the other hand, typically will attempt to 
identify the processes necessary for the production of an 
output and their interrelationships in time. 

2. The output map is focused on demonstrating tho functional 
dependencies between three classes of outputs (i.e., 
products, events and conditions), not their dependencies 
in terms of production schedules or time deadlines . 

3. The output map is developed generally through a series of 
stages, the initial output map being generated while the 
initial site-visitation team is onsite. This enables 
direct interaction to take place between the project staff 
and the initial site-team. Some beginning elements of an 
output map have been recognized prior to an initial site 
visit. The documentation available for review prior to 
site visits is , for the most part , not (^apab 1 o n f provi d i up, 
the information necessary to derive a representative output 
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FIG. 8. Output map derived from tentative output index 





FIG. 9. An output map developed prior to the initial site visitation 
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map. The initial output tnap will be refined, modified, 
and finali.'^ed in a series of stages discussed later in this 
volume . 

The Elements of a Map 

A number of elements can be identified within the illustrations 
of output maps (Figures 8, 9, 10 and 11); solid line boxes; labels; 
code symbols; horizontal lines; vertical lines; brackets; dotted lines; 
dotted line boxes; and vertical (long and short dash) lines. Each of 
these elements contributes to the total information contained in a map. 
The following paragraphs identify the information presented by each 
element . 

Solid line boxes 



I I Each solid line box represents a specific output 

that the project is seeking to achieve* If the 
box stands alone (is not connected to any other 
box by a line) one of two conditions exists: 

(_a) the output is considered to have value, but is 
not related to any other output, or (^) the output 
index did not contain output identifications that 
allowed other outputs to be linked to it. 

Labels and 
Coded Symbol 

Within each box there is a label which is the 
descriptor of the output represented. The labels 
found in the box are the same ones used to describe 
a particular output throughout the profile. 

Each box contains, along with the label, a coded 
symbol. Each symbol is composed of a letter which 
identifies the structure of the output (P-product, 
E-event, C-condition) , and a 2-digit numeral which 
identifies the output sequentially with respect to 
the ether outputs in the same profile. These labels 
and coded symbols can be found in the Output Index. 

Horizontal lines 



Horizontal lines between boxes indicate that the 
outputs so connected have side-effect relationships, 
that is, the production of one influences the other, 
and vice versa. 



P-01 Final 
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Vertical lines 



Vertical lines between boxes indicate that the upper 
output is dependent on the lower. Until the lower 
output is completed the upper one cannot be completed. 
In the total map, boxes connected by vertical lines 
are hierarchically arranged, those at the top of the 
map being dependent upon all those below. 



Brackets 



Boxes enclosed by brackets represent outputs which 
are influenced by, but not dependant on the output 
represented by the box linked horizontally to the 
bracket(s) . 



Dotted lines 

Dotted lines connecting two brackets are used to 
indicate outputs that are influenced by another 
output when the outputs encompassed by a bracket 
area are large in number, or when influence is 
carried across more than one page of the map. 

Dotted line boxes 



Dotted line boxes represent those outputs which 
are either generated outside the project, but 
influence it , or are outputs generated by the 
project as a function of other outputs but have 
not been indexed by project staff. 

Vertical (long and short dash) lines 

I When more than one page is needed to display an 

! output map, vertical lines (long and short dash) 

-I are used on the right of the first page and the 

I left of the second page to indicate the point at 

which the two pages coincide. 

Under unusual circumstances some outputs may appear more than 
once in a map. This results when they are related to other outputs 
in different ways, i.e., dependent on one set but influenced by 
another. When it is impossible to display both the relationships by 
one placement, outputs are repeated. 

Once the purpose of an output map is understood and the various 
elements within it are defined, the reader should be able to extract 
a great deal of information from a careful analysis of a map. 

In summary then, an output map is a visual, schematic representa- 
tion of a series of dependencies among outputs of a particular site. 

The function of an output map is to visually represent the interdepend - 
encies of outputs and not the time sequence required for their produc- 
tion. It is therefore not process oriented and, by similar token, 
time is an incidental element and not the focus of the map. An output 
map usually has as its focus one or more contracted-for outputs (and/or 
outputs which have a degree of significance). This can be understood 
more clearly if the reader will review the 20 output maps available in 
Volume IV of the Oregon Studies. It is not often the case that the out- 
put map is capable of representing the entire scope of any one project. 
Instead, it attempts to capture a significant and representative sample 
of the outputs being produced by each pMject. 
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It is critical that the output map be developed at least in an 
initial stage prior to the initial site-visitation team^s briefing 
of the data collection team. The output map provides an interview 
orientation that is critical for the data collection team. It enables 
the data collection team to understand the dependencies of outputs one 
to another and to understand the function or role of the individual 
staff members on any particular site, and it provides some clarity as 
to the outputs the site feels are focal. Given this set of sensitiv- 
ities, the data collection team is able to focus much more rapidly 
with an individual interviewee in terms of that individual's contribu- 
tion to the successful accomplishment of the outputs in question. 

The output map is not only a vehicle for displaying the inter- 
dependencies of outputs of any particular site, but is a tool to be 
used by the data collection team in attempting to learn as much about 
a particular site as possible in a short period of time. By providing 
a vehicle for focusing an interview and for coming to grips with an 
understanding of the site in question, the output map enables the data 
to be collected in a cost-effective manner. 



Activity Block 3: Interviewer Training 



Output Index : 

E-14 Study of Onsite Data Gathering Operations (facilitating, 
production) 

E-15 Group Discussion of Onsite Data Gathering Operations 
( facilitating , production) 

E-16 Demonstration of Interview (facilitating, production) 

E-17 Critique of Demonstration (facilitating, production) 

E-18 Practice Interview (facilitating, production) 

E-19 Critique of Practice Interview (facilitating, production) 
E-20 Practice Recapping (facilitating , production) 

E-2l Critique of Practice Recapping (facilitating, production) 
C-22 Skilled Interviewers (component, production) 

E-23 Team Exercise (facilitating, production) 

E-24 Critique of Team Exercise (facilitating, production) 

C-25 Ability to Work as a Data Collection Team Member 
(facilitating, production) 
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Output Map ; 




The primary mode of data collection used in this me thodology is 
one-to-one ir ^erviewing. (One interviewer to one interviewee). The 
gathering of data ^rom a site normally requires several interviewers. 

With one of their number designated as leader, they act as a data 
collection team. Individually and collectively the interviewer's ex- 
pertise is essential to the successful implementation of this method- 
ology. 

Training an interviewe r, there fore , involves both the individual 
and the team. 

The subject matter of interviewer and team operations is set 
forth in considerable detail in Activity Blocks 2, 5, b, 7, and 8. 

Rather than repeat that detail, this section will deal with techniques 
of getting that subject matter across to the potential interviewers. 

The achievement of methodology objectives appears to be best 
accomplished when there is considerable understanding by all project 
personnel of the interrelationships and interdependencies of all 
procedural steps and actions. Only such understanding by all personnel 
can provide the basis for flexibility in selecting data collection teams 
to insure that the data gathered will be in a form compatible with sub- 
sequent procedural steps. 

Interviewer training should be conducted by a person highly knowl- 
edgeable about interviewing techniques in general , about the kinds of 
data sought with this methodology, and about the basic concepts underlying 

51 
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Liu* t ud i * It would be desirable for tho Interviewer Trainer 

to have experienced the application of this me tiiodo logy . However, 
studying appropriate volumes of the Oregon Studies and participating in 
the suggested learning experiences should prove to prepare an individual 
to be able to fill the role of Interviewer Trainer. 



H-14 Study of Onsite Data Gathering Operations , and 
E-15 Group Discussion of Onsite Data Gathering Operations 



The textbook recommended for interviewer training is this method- 
ology manual. In addition, each trainee should be given a supply of: 



Forms ^ 

03 - Job /Task Inventory 

04 - General Position Activities Questionnaire 
05-A - Output Recap 

05--B Output Recap 

06 - Interview Recap; Standards 

07 “ Interview Recap: Tasks 

08 - Interview Recap ; Enablers 

Worksheets 

11 - Contextual Cue 

13 - Training Suggestions Recap 



It is further recommended that each trainee be issued a portable 
battery powered tape cassette recorder^ and a supply of tape cassettes. 

Activity Block 5 should be thoroughly studied by each trainee. 
Subsequent group discussion led by an Interviewer Trainer will provide 
an opportunity for individuals to elaborate their understanding of not 
only how certain steps are to be performed but also why they are per- 
formed. It should be anticipated that several group discussions will 
be required. 



£-16 Demonstration of Interview , and 
E-17 Critique o f Interview IV n ons tration 

A demonstration of inter iewing should be conducted by the Inter- 
viewer Trainer. Trainees should be especially observant of the demon- 
strator's focusing of the interview on selected outputs and his atten- 
tion to refraining from leading the interviewee's responses. The 
demonstration should be conducted without interruption for questions 
and should be tape recorded (as would an actual interview). Follow-up 



See Volume 3, Schalock/Sell paper 

^ Sample forms and worksheets are found in Appendix 1 . 

It is recommended that a type of recorder be used which gives a 
visible or audible signal when a side of the tape has been completed. 



discussion of the demonstration should be accomplished as soon as pos- 
sible using the recording of the demonstration as the basis for ques- 
tions, cinswers ^ emphasi zing, and cri tiquing. (In the interest of con- 
serving time, the demonstration may be limited to eliciting data 
around only a single output.) 

E-18 Practice Interviews , and 
E-19 Critique of Practice Interviews 

Following the above demonstration-discussion sequence several 
practice interviews should be conducted by each trainee. Initial 
trials should be limited to eliciting data around a single output 
with subsequent efforts extending around two or more outputs. The 
most desirable procedure would be for trainees to practice interviews 
with people within their own agency, but outside of the immediate 
pro j ect staff , To ligh ten the burden of such demands on outside 
manpower, an alternative would be to have practice interviews con- 
ducted by two trainees, ea"h of whom would handle part of the later- 
view. Least desirable, but still functional, would be to have 
trainees conduct practice interviews with members of the immediate 
project staff. 

All practice interviews should be tape recorded. Critique '^f 
these practice interviews should be conducted by the Interview 
Trainer on an individual basis, with both trainer and trainee to- 
gether listening to the tape recording, identifying errors, and dis- 
cussing problems encountered. If the trainer detects interview 
problems that consistently occur among his trainees, group sessions 
can be called to resolve these difficulties. 

E-20 Practice Recapping , and 
E-21 Critique of Practice Recapping 

0ns ite data reduction , another responsibili ty of all team mem- 
bers, is described in detail in Activity Block 7 of this volume. This 
activity does not lend itself well to demonstration. However, an 
eff^ * Ive training technique is for the trainee to recap his own prac- 
tice interviews. This should be done following the interview critique. 
Again, upon completion of the practice recapping, a critique between 
the Interview Trainer and the trainee ( interviewer/recapper) should 
begin to establish the needed expertise. If necessary, additional 
practice interviews , practice recapping , and associated cri tiques can 
be scheduled. 

C-22 Skilled Interviewers 



The events discussed previously should result in the condition of 
skill on the part of interviewers. This is one of the major anticipated 
results of this block of activities. The subsequent outputs cannot be 
effectively accomplished unless interviewers individually possess the 
skill to ob tain the required data in the requi red fonTi, 
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Jn addition to interviewing and recapping, for which the foregoing 
training will equi]» team members, the team Leader carries additional 
responsibiliti.es, and thus needs additional expertise. As primary 
spokesman for the data collection team, he must effect the initial in- 
terface between the team and the site personnel, 

Jt is recommended that the team leader be a senior member of the 
project staff. He must be capable of explaining the purpose and scope 
of the team^s effort. If he is personally acquainted with the site 
personnel, he has a distinct advantage. In conducting the initial meet- 
ing with the site personnel, his "se tting-of-the-s tage'* f jr the team’s 
activities can do much to insure their success. 

Many duties of a team reader are of an intangible nature and may 
well be a matter considered in selecting team leaders, rather than a 
matter of specifi:: project training. Onsite, the team leader acts in 
the capacity of team coordinator. Hs must be cognizant of the kind and 
extent of data that his team will collect, and must take steps to insure 
that the data are complete before tha team departs from the site. He 
must be aware of those portions of the data that result from combined 
team efforts and must call and lead meetings of the team to collect 
those data. He must maintain an awareness of any developing personal 
or rapport problems, and offset these in a manner advantageous to the 
data-gathering effort. He provides onsite quality control. 

The team’s efforts as a team can be exercised through an actual 
or simulated experience. Preferably, a local project can be used as a 
practice site. If the practice site data c^'^n later be included in the 
final resulting data compilations, so much the better. In providing 
such an experience-, every effort should be made to provide the entire 
process from team briefing through site visit and onsite data reduc- 
tion for each team. It is preferable, if realistic conditions can be 
maintained, to utilize each team throughout an entire experience. It 
is less desirable to have other teams observing a team in action than 
it is to provide such experiences for all teams in training, with post- 
experience staff discussion sessions . During the team experience and 
post-experience sessions, it is recoinmended that the Interviewer 
Trainer play a decreasingly prominent role, allowing the team to 
develop the necessary initiative to face and solve problems as they 
are encountered. 

C-25 Ability to Work as a Data Collection Team i^ember . 

The ability of a data collection team to coordinate their indi- 
vidual efforts and function as a purposeful entity is essential to 
the production of useful data. One or more data collection teams are 
a necessary part of this methodology. The composition of a team does 
no t have to remain cons tan t . If the individuals are competent and 
compatible, a workable team can be formed specifically for obtaining 
data from a specific site. The team’s size, its leader, and its 
appearance can be varied to fit the anticipated site situation. Its 
essential elements are: capable individuals, a capable le ider, and a 

cooperative responsible team approach . 
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Quality Control 



To insure rigid adherence to procedural standards, the training 
process must be ongoing. The Coding Resolution Team reviews and works 
with all recapped data during the ongoing project. Their quality 
checks quickly determine areas where deficiencies exist and point out 
requirements for further training of active team members. 



Activity Block 4; Preparing a Data Crllection Team. 



Output Index : 

E-26 Composing a Data Collectica Team (facilitating, management) 
E-27 Briefing a Data Collection Team (component, management) 

C-28 Data Collection Team Prepared for Site Visit (component 
production) 



Output Map : 
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E-26 Composing a Data Colle»:tion Team 



Upon completion of the initial site visit a data collection team 
must be selected from the investigating agency *s staff. One of this 
team should be a senior staff member who is designated as the team 
leader. These are the people who will spend time at the project site 
to interview staff members about their activities. In determining the 
number of interviewers needed to compose the team, the following should 
be considered. 



1. It is essential that the number of staff members who com- 
prise the target project staff be known. It is also necessary 
to know the outputs for which each of those staff members is 
responsible. It is through this combination of outputs and 
staff members that a determination can be made in terms of the 
number of interviewers necessary to make the site visit. 

2. A general guideline that may be applied in determining the 
number of interviewers needed includes the following factors : 

(a) Tn one 2 hour session with an interviewee, three 
outputs can be successfully interviewed about to 
the depth of detail required by this methodology; 

(b) Within a i day period of time, one i ntervi.ev.^er 
can interview cwo project staff members ; 

r 
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(c) Approximately A to 6 hours of time will be 
required for the interviewer to recap and summarize 
the data collected from each interviewee; 

(d) Depending upon the vagaries of scheduling on any 
particular site, it is usually unreasonab le to 
expect any single interviewer to deal with more 
than four staff members in i week; 

(e) In any eventuality, no fewer Lhan two staff members 
should visit any site as a team. 

Of course, the size, scope, or nature of some projects may 
require modification of the ratios indicated above. 

3. Any sensitivities or impressions derived from the initial site 
visit may also be utilized to facilitate the final team selec- 
tion. This calls for a fairly complete knowledge of the in- 
dividuals available as team members in terms of not only 
professional expertise but their personal biases, political 
viewpoints, in short, anything of a sensitive nature which 
may prove upsetting or particularly appropriate to the staff 
of the site to be vis i ted. 

4. The general intent in making this team composition is to 
cluster a group of competent individuals who can enter a 
particular site, create as little disturbance as possible while 
on that site in terms of disruption of schedules or the 
raising of controversial issues, and leave that site essen- 
tially in the *'condi tion'* in which it was found. it is dif- 
ficult to elaborate on this point because of the immense 
variety of personality characteristics that can be involved. 
Suffice it to say that this point is recognized as a signif- 
icant consideration in the methodology, and the time spent 

in attempting to understand individual staff members and the 
reaction the site may have to them is an investment worth- 
while to the data collection process . 

£-27 Briefing^ a Data Collection Team 



In order to transmit to the members of the data collection team 
the information which has been collected by phone and personal visit, 
a team briefing is conducted- This is preferably done 1 week prior 
to the scheduled site visitation. During this team briefing, the 
following events should occur: 

1. The sets of data which have been collected by the initial 
site-visitation team are shared with the data collection 
team. 

2. Members of the data collection team have an opportunity at 
this time to sensitize themselves to the various aspects 

and dimensions of the site which they will be visiting. These 
sensitivities can be obtained not only from a review of the 
summarized data brought back from the initial site visitation, 
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but also through a question and answer session with the per- 
sons who made the initial site visit, 

3. In addition to the above information sharing and sensitiza- 
tion, the determination of outputs to be interviewed around 
and a preliminary assignment of interview staff to interviewees 
is also made . Particular attention is given to assigning the 
interview staff to outputs with which they have some familiarity. 
Sensitivities brought back by the initial site team as to align- 
ments of interviewee /interviewer personali ty characteristics may 
also help to shape these assignments, 

4. Since total interview time usually will be limited, a selec- 
tion of the outputs is probably necessary at this point. 

Logistic concerns, therefore, provide the first criteria in 
final output selection, A second criteria is the purpose for 
which the methodology is being employed. If mapping or de- 
scribing a domain is the purpose, a broad range of differing 
outputs may be desired, and selection made to cover as many 
different outputs as possible. If, however, a detailed study 
of a particular part of the domain is the purpose, then those 
outputs bearing on that area would be selected. It should be 
noted, however, that a detailed study of only a portion of 

the domain may be invalid if the related context is not under- 
stood, For example, if the emphasis is to be on management, 
it may be necessary to interview (though to a lesser degree) 
around production efforts to provide the context in which the 
management occurs . 

5. Regardless of the criteria used for selecting outputs to be 
interviewed around, the data collection team muse be made to 
understand those criteria, so they share common objectives 
during the visit. 

6. In the event that not all of the site's outputs will be the 
subject of interviews, or if fewer than the total staff of 
any particular site are to be interviewed, the responsible 
site person is usually notified prior to the team visit as to 
the nature of this selected set . 

C-28 Data Collection Team Prepared for Site Visit 

The data collection team, as a result of this meeting, should be 
as familiar with the site as time and data will allow prior to any 
site visit. Data from Forms 01 and 02, as well as the preliminary 
forms of the contextual map, output index, and output map are dis- 
cussed with the team. Any sensitivities which have been picked up by 
the initial site-visitation team are shared. 

Prior to the site visit, the Coordinator of Data Inputs will pro- 
vide the team of interviewers with a package of materials necessary for 
the collection of data from that site. A package of materials is made 
up and provided to each individual team member. This package includes: 
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• i;|i- L(^“(Jciic « «>|;y i)i l^orni 01 (l^rojeci CoruacL iUrcurd) . 

0 Completed copy of Form 02 (Project Questionnaire) . 

• Sufficient number of Forms 03 (Job/Task Inventory) and 04 
(Genaral Position Activities Questionnaire) for use with 
each interviewee. 

• Form 05-A (Output Recap). 

• Form 05-B (Output Recap) . 

• Enough Forms 06 (Standards Recap) , 07 (Task Recap) , and 

08 (fabler Recap) for the interviewer to recapitulate the 
verbal information from each of his interviews to a format 
suitable for coding, storage, and later use. 

0 A copy of selected documents and materials collected from 
the project site to date. 

0 A copy of the initial output map, output index, and contextual 
map for the project. 

0 Worksheets 11 (Contextual Cue), 12 (Interview Record Form), 

13 (Training Suggestions Recap), and 14 (Interviewer Notes 
on Interrelationships) for use with each interviewee. 

0 Worksheet 17 (Checklist of Project Data Inputs). 

0 A tape recorder, tapes, and batteries. 



Activity Block 5: Site Visit by a Data Collection Team 



Output Index : 

E-29 Team/Site Conference (component , production) 

P"30 Selected Outputs for Interviews (component , production) 
E"31 Interviews (component, production) 

Output Map : 
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E-29 Te am/Si te Confe re nee 



As a vehicle for introducing the site staff to the interviewers » 
an initial team/site meeting should be scheduled for the first day 
the team is onsite. 'Hie following items need to be considered during 
this meeting: 

1. It is the Team Leader’s responsibility to conduct this session 
calling on the team members for comment when appropriate. 

2. Outputs selected for interview concentration are reviewed for 
the site personnel. 

3. A second draft, where appropriate, of the output index and out 
put map are shared and verified with the site personnel. 

4. If Form 03 and 04 were not initiated on the initial site visit 
then their nature and importance are explained. These forms 
are distributed to the entire site staff. If completed ouring 
the conference, interviewers examine them quickly to assure 
completeness and compliance with directions for respondiiig. 

5. One of the most important items to be accomplished during this 
team/site conference is the arranging of tentative interview 
schedules for the duration of the time on site. This is 
usually handled directly by each individual interviewer as a 
subpart of this session. As mentioned before, the minimum 
interview time is usually stated as 2 hours, with a possible 
maximum interview time of 4 hours. 

6. As sometimes happens, certain details on Form 03 may not have 
been readily available before (e.g., data that needed "looking 
up") . This should be either filled in now, or a definite 
arrangement should be made to collect the missing data befere 
the team departs. 

P-“30 Selected Outputs for Interviews 

Selecting the outputs to be used as the subject of interviews goes 
through several stages: 

1. The initial site visit provides an output index and an output 
map. These are tentative in nature . 

2. These documents are provided to the data collection team 
during the team briefing and serve as the subject matter for 
group discussion leading to the identification of outputs 

to be subjected to interviewing. 

3. The team/site conference provides a significant opportunity 
to discuss and amplify the output index and map. Addition 
of outputs at this time will warrant further attention to 
the selection of outputs for Interviewing focuc. 

4. Normally, a team discussion following the team/site conference 
will formalize a set of outputs for interviews, and serve to 



finalize a. strategy for addressing these outputs during the 
interviewing. 

At this stage the output map should be substantially complete , but 
still open, in the event that new outputs emerge during interviewing. 

E“-31 Interviews 



The interview is the primai^r data-gathering tool of this method- 
ology. Its manner of employment is critical to the validity of the 
data to be gathered. The prime purpose of the methodology is to gathev 
data that portrays accurately the processes and competencies needed to 
produce outputs, as communicated by the individuals who produce them. 

The descriptions of what must be done and how it is done are elicited 
from those most directly involved, stated in their own terras with a 
minimum of prompting or suggestion. The interviewee is considered to 
be an expert in his own work, and the interviewer's efforts are focused 
on getting the interviewee to explain his work so that others can un- 
derstand the standards of acceptability held for an output, the tasks 
necessary in producing that output, and the enabling knowledges, skills, 
and sensitivities requisite for generating such an output in the proj- 
ect context. 

Definitions : 

0 Standard - A specific criterion applied to, excellence expected 
of, or a criterion by which judgments of adequacy 
are made about an output or the processes which 
generate it. 

• Task - A unit of work performed in producing a specified 

OUTPUT to a specified STANDARD. 

• Enabler - KNOWLEDGES, SKILLS, and SENSITIVITIES needed to 

produce a particular output. 

• Knowledge (as Enabler) - That which is to be known as a 

requisite to the accomplishment 
of an output. 

• Skill - An ability, proficiency or expertness considered 

requisite to the accomplishment of an output . 

0 Sensitivity - A specific perceptiveness and responsiveness 
considered requisite to the accomplishment 
of an output. 

The following points should be remembered in conducting the on- 
site interviews: 

1. Since the portable battery-powered tape recorder is used, 

the interview could conceivably be conducted anywhere. How- 
ever, an office in the normal context of the site is con- 
sidered the most appropriate setting. The choice of inter- 
view setting may be a good opening gambit to get things 
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started. Since the interviewer is charged with obtaining 
contextual information , the interviewee ’s own office as an 
interview setting may yield clues to his personality or 
to the conditions under which he normally functions. 

2. The interviewer's demeanor and conduct can make or break the 
'Success of an interview. He needs to tread a narrow path. 

Not only should he convey an impression that he is competent 
(and in that way enhance the image of the team and project), 
but he should simultaneously convey that the interviewee's 
expertise provides essential data and will be accurately and 
unders tan dab ly gathered. 

3. The interviewer should be alert to every word and inflection 
used by the interviewee, framing his questions to clarify his 
own unders landing of the interviewee 's remarks . The inter- 
viewer should convey a genuine interest in the interviewee 's 
work, and try to follow the course of the conversation unerr- 
ingly to the interviewee's meaning. The interviewer should 
then evidence his understanding of the information elicited 
through clarifying ques tions and/or occasional paraphrasing 
of the data. 

4 . Regardless of the interviewer's own expertise , biases , or 
limitations, he should gain the information sought from the 
interviewee — unclouded by the interviewer's beliefs and 
opinions. The data being sought throughout this methodology 
is descriptive in nature. No attempt should be made to eval- 
uate. No judgment should be made or reported. No feeling of 
approval or disapproval should be conveyed to any interviewee. 

5. Assuming that the initial conference between the data collec- 
tion team and .the site personnel has transpired according to 
plan, the interviewer should have: 

(^) Been introduced to his interviewees. 

(M Confirmed the outputs and the interviewees who are 
most knowledgeable about those outputs. 

(c) Confirmed the outputs around which he is to focus his 
interviews . 

(^) Made appointments for interviews. 

(^) Studied any available vita information on his inter- 
viewees . 

(_f) Discussed, with team members, any known personality 
factors concerning his interviewees. 

6. Armed with this knowledge, the interviewer should arrive 
for his interview appointment on time. It should be noted 
that the data collection team's very presence on the site 
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is costly to the site in terms of disruption to the daily 
routine and in terms of manhours of interview time. The 
team has been allowed to come in to gather data. If any 
modification to schedules is to be made, let it be instigated 
by the site. A serious businesslike approach by all team 
members creates an impression of professional competence and 
can be initially displayed by meeting interviewees at the 
agreed-upon place at the appointed time. 

A by-product of such conduct is reciprocal behavior in 
which the interviewee will extend a like courtesy, thus 
rendering the interviewer's own use of time productive. The 
importance of such conduct cannot be overemphasized. It is 
natural that superfluous behavior occurs, especially when 
interviewer and interviewee share interests in common . How- 
ever, to use the committed time for such purposes may ulti- 
mately preclude gaining audience for an unanticipated follow- 
up. 

7. The words an interviewer uses in greeting and establishing 
rapport with an interviewee are a result of his own personality 
and interpersonal skill. At a minimum, the team member should 
be ready to more fully explain the project objectives and the 
team's activities if the interviewee indicates a desire for 
this sort of information. The interviewer plays his role 

"by ear" being ready with "small talk," the local political 
situation, the weather, or whatever the situation demands, 
keeping "one eye on the clock." 

8. Transition from the small-talk stage to business can be accom- 
plished behaviorally by setting up the tape recorder, and at 

a judicious point indicating to the interviewee (again, if 
mentioned earlier) that the recorder is being used so that 
his exact words will be captured and his time will not be 
wasted by longhand note taking. 

9. Initially, the interviewer should focus the interviewee's 
attention on the outputs to be discussed, and confirm that 
the interviewee is actually knowledgeable about those out- 
puts • 

10. It is frequently the case that interviewees are so familiar 
with their work that some outputs are taken for granted and 
do not appear as identifiable outputs until they describe the 
tasks they perform. The alert interviewer can often identify 
additional outputs through his questions which seek clarifica- 
tion. 

In this situation, it is advisable for the interviewer to 
make the decision himself as to whether or not to focus the 
interview on a newly identified output. He must consider in 
his decision: (a^) the value of that output to the study, 

(^) the available time for interview and/or the advisability of 
continuing the interview beyond the planned time, and (c^) the 
possibility of arranging a second interview. 
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Output selection can be finalized only upon completion of 
the onsite visit, although the list of outputs approaches fi- 
nality with each interview. 

11. In relation to any output around which an interview is focused, 
the data sought are: 

(^) Standards by which the acceptability of the output is 
judged. 

(b) Tasks involved in the successful accomplishment of the 
output . 

( c) Knowledges requisite to the success f u 1 accomplishment o f 
the output. 

{d) Skills requisite to the successful accomplishment of the 
output . 

(e) Sensitivities requisi te to the successful accomplishment 
of the output . 

12. These data may be elicited either by asking questions directly, 
or by more circuitous conversation about the output, mentally 
noting that the data items have or have not yet been obtained. 

The degree of directness is a matter of interviewer style, and 
each extreme has its pitfalls. In using the direct type of 
approach, the interviewer should be extremely cautious in his 
use of words to insure that he doesn’t lead the responses of 
the interviewee. For example, in determining standards for a 
document, the interviewer could properly ask, **If you had this 
finished document in front of you, how would you know that it 
was acceptable?" The interviewer could not properly ask, "How 
would you measure this document’s acceptability?" In the lat- 
ter case, the interviewer led the interviewee by suggesting 
that some measurement might be used. The question of measure- 
ment could be properly used only after the interviewee had 
indicated, in response to "how would you know it was acceptable?", 
that some measure or test of the document was actually made. In 
other words, the interviewer can follow a lead for clarifica- 
tion , but he does not do the leading . 

An interviewer who follows a less direct approach — perhaps by 
asking the interviewee to generally describe the output — 
runs the risk of missing important items of data. The approach 
an interviewer uses (direct or indirect) will become a matter 
of personal style and probably will be somewhere between the 
extremes cited above. 

13. Style of interview may be influenced by the kind of output 
that is the subject of the interview at the moment, as well as 
by the kind of responses evoked from the interviewee — the 
interviewee’s personality, or his way of looking at the world. 



Thus the kind of interview resulting in the most effective data 
gathering will be a combined outcome of interviewer personality, 
interviewee pe rsonall ty , and the subject of Interview (output 
being addressed) . 

For example, when interviewing about either a condition 
or an event, rather than a product, as a form of output, the 
interviewee may see what he does as a process, rather than 
something that produces an output or end result. For example, 
staff morale may be "doing things to keep people happy." The 
interviewer needs to be sensitive to such viewpoints and should 
not try to "make them over" in the interviewee's mind. 

A helpful way to handle this is for the interviewer to have a 
clear ordering in his own mind about what constitutes a 
valid output , in the form of a product , event , or condition , 
and then the relationship of tasks, standards, and enablers to 
that outcome, so he can recognize in his own mind when he has 
obtained answers that establish these relationships. 

14. Following through to get the necessary responses can be facil- 
itated by making notes, by keeping a mental checklist, or by 
filling in gaps at successive interviews, where more than one 
interview with a specific interviewee is scheduled. In the 
last mentioned case, the process of recapping and contextual 
debriefing (Activity Block 7) can serve to detect the gaps 
that may have been missed in interviewing. 

15. Since the data is reduced and reported in the order of stand- 
ards first, tasks second, and enablers (knowledges, skills, 
sensitivi ties) third, the preferred order of questioning 
follows that pattern. An additional reason for questioning 
in that order is that by considering standards, the inter- 
viewee may be alerted to tasks he performs in achieving the 
standards of quality as well as the tasks performed in produc- 
ing the output. 

16. The basic interviewing questions around any output are: 

How do you know that the output is acceptable? 

How do you produce the output? 

What do you need to know to produce the output? 

What must you be able to do to produce the output? 

To what must you be sensitive in producing the output? 

Do not anticipate that the data will be forthcoming in 
neat, orderly fashion. Few interviewees are able to be com- 
plete in their responses in any one of the above categories . 

For example, suppose that standards for the output "inter- 
view data" have already been elicited, and tasks are now 
being sought. The interviewee might mention that he performs 
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a reliability check to determine that all his interviewers 
were obtaining answers to the same questions (a task) . How- 
ever, no standard of reliability had been mentioned. The 
alert interviewer should then inquire as to what degree of 
reliability is desired; and hence, is that degree of reliabil- 
ity actually a standard for the acceptability of the ’’inter- 
view data”? In a case such as that just used as an example, 
a typical interviewee response is, ”0h yes, of course. I 
forgot to mention that one when I was telling you about stand- 
ards . ” 

The above phenomenon occurs frequently throughout the 
interview. In eliciting skills, the interviewee frequently 
comes up with additional tasks. Questioning for sensitivities 
frequently results in addi tional standards — particularly o f 
the "I just feel that it is done right” kind of standard. 

It takes extreme concentration on the part of the interviewer 
to detect cues whi .h allow him to follow and probe for the 
data he seeks. 

17. The most difficult aspect of interviewing in this methodology 
is the interviewer * s knowing how far to probe or just when he 
has achieved the degree of detail that can be adequately 
handled (or is desired). A number of small indicators combine 
to provide the interviewer a ”feel” or "sense” in this respect. 
In determining standards, for example, the interviewer’s knowl- 
edge of similar outputs may give him a clue as to what the 
interviewee might possibly say. By considering such a hypo- 
thetical s tandard in advance of the interview, the interviewer 
may be able to form an idea of the degree of detail he wishes 
to attain. A few opening questions to elicit the interviewee’s 
description of the output in question may provide insight into 
his normal way of speaking, and hence indicate to the inter- 
viewer how much probing will have to be done. 

Basically, the interviewee’s statements of stajidards and 
enablers are accepted as stated; however, probing to understand 
the interviewee’s exact meaning should provide the detail re- 
quired. The same problem of degree of detail is even more 
challenging when eliciting tasks. The definition provides 
little help (a unit of work...), in that no scope of "unit” 
is provided. Taking the definition as it reads, the inter- 
viewer can only understand that the size or complexity of a 
task can vary. Here again, a feel or sense must be developed. 
Tasks should be meaningful units of work; not so finite as to 
constitute a single homogeneous action of the worker (e.g., 
pick up pencil) , but rather a composite statement describing 
a unit of work that is meaningful in the eyes of the inter- 
viewee (e.g., "Write outline of Chapter Six"). Verbs such as 
supervise, insure, oversee, manage, etc., however, are so 
gross in scope that many meaningful units of work might remain 
hidden within. Probably the best guide is to probe until 
either the interviewer believes that he fully understands the 
action represented by the task or he has detected that the 
interviewee is having difficulty in being more specific. 
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The following example illustrates the use of nrobes to arilve 
at an acceptable level of detail, (Tliis is an edited transcript 
of an actual interview about the output "Atmosphere for 
Constructive Criticism, ") 

Interviewer: How do you go about creating an atmosphere 

of constructive criticism? 

Interviewee: We have to start, of course, with well 

qualified people, ITien through direct 
interaction we establish the atmosphere 
of cons tructive cri tic ism. 



Interviewer: Do you mean that you personally critique 

the work of your subordinates? 



Interviewee : That» certainly » but much more than that. 



I do interact with an individual to critique | 
his documents,! However, it is a mutual 



kind of thing — |l ask him to critique my 

too^l In this^ way we work together to 
achieve better products. 



In this example, the first interviewee response lacked suf- 
ficient detail. The second response shews two tasks (in boxes) 
that are at an appropriate level. Probing questions could be 
asked which would lead to more definitive statements, e,^,, 
"Call staff member to iny office" — "tell him his sequence needs 
improvement" — etc. However, these statements are more detailed 
than necessary. 

18, Standards, tasks, anjjJ^ablers are determined in relation to 
each of the assigned outputs , During the interview, notes 
can be made, if the interviewer needs such "memory joggers," 
on blank paper or on a contextual cue sheet (Worksheet 11), 
However, such notetaking should be minimized, as the essential 
data is recorded on tape. 



19. When the data relevant to a specific output has been gathered, 
the interviewee's suggestions or ideas are requested as to how 
best to train people to do the kind of work he has described. 
Cues for starting this line of questioning may have been noted 
from the interviewee's previous responses about knowledges, 
skills, and sensitivities. His recommendations may stem from 
his own background and the problems he encountered in becoming 
qualified to produce or assist in producing the outputs he is 
knowledgeable about, or they may stem from his experience in 
trying to locate adequately qualified staff. In some cases, 
his training recommendations may have been tried out in an 
attempt to increase the capability of professionals already 
on the payroll. As in all facets of the interview, the inter- 
viewee 's thoughts and ideas are the data sought — not the 
thoughts and ideas of the interviewer. Therefore, extreme 
caution must be exercised to refrain from prompting or sugges- 
ting answers to the interviewee. 
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It is very possible that an interview may require in excess ot‘ 

2 hours. The questioning has been detailed and has required 
considerable attention and serious thought. The interviewer 
should be alert for signs of fatigue on the part of the inter- 
viewee and if advisable, schedule a second Interview later. 

With the gachering of the training suggestions, tlie data sought are 
complete. In terminating the interview, however, it is advisable to 
’*keep the door open*’ for a brief return interview if any gaps in the 
data are seen during the initial data reduction (recapping the tape 
recordings of the interview). No interview should be terminated with- 
out expressing appreciation for the time, thought, and effort given by 
the interviewee. The usual experience has bean that both parties ex- 
press a sense of having found the interchange beneficial. Invariably 
interviewees have found that responding has sharpened uheir own percep- 
tion concerning their own work. 

Immediately after departure from the interviewee's presence, the 
interviewer should note any items of contextual importance. A copy 
of Worksheet 11 can be used for this purpose. These notes, made while 
the impressions are fresh in mind, are helpful during daily team de- 
briefings and during a final team debriefing. (See Activity Block 7 
for contextual debriefing.) 

The following is a procedural outline for interviewing that is 
recommended for this methodology. 



1. Arrive for interview at appointed time. 

2. Greet interviewee. 



3. Turn on tape recorder. 

A. Confirm interviewee's relationship to selected outputs. 



5. About each output: 

(^) Question interviewee about 
bili ty . 

(^) Question interviewee about 
(c) Question interviewee about 
(^) Question interviewee about 
(O Ques tion interviewee ab out 
(_f ) Que s t i on in t e r vi ewe e ab o u t 
training. 

6. Terminate interview by thanking 



standards of output accepta- 

tasks • 
knowledges . 
skills . 

sensitivities . 

his recommendations for 

interviewee for his cooperation. 



The data obtained by interview, at this point in time, is in the 
form of a tape recorded conversation. It contains much of the desired 
Information. In this form it does not lend itself to easy processing 
and analysis. Subsequent processing of the data will extract the 
essence of the conversation and reduce it to more appropriate form for 
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analysis. However, in this highly unrefined form (tape recordings of 
conversation), the data represent one of the major outputs of an employ'- 
ment of toe methodology. 



Activity Block b: Questionnaire Administration 



Output Index : 

E-32 Delivering the Questionnaires to Respondents (facilitating, 
management) 

E-33 Explaining the Questionnaires to Respondents (component, 
production) 

E-34 Collecting the Ques tionnaires from Respondents (component , 
production) 

O utput Map : 



E- 
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E-32 Delivering the Questionnaires to Respondents 

It is critical that Forms 03 and 04 be completed by all site per- 
sonnel (not only interviewees) by the time the data collection team 
departs from the site. 

Responsibility for the delivery of Forms 03 and 04 resides with 
the leader of the initial site visit and/or the data collection team. 

A feasible time for distribution of these forms is the team/site con- 
ference. If the total site staff has been assembled, rationale and 
instructions for completing these two questionnaires can be presented 
economically. In practice, it has been found to be necessary, at 
times, to request that an interviewee complete the forms as the initial 
step of an interview. In such case the interviewer must assume seme 
responsibility for questionnaire administration. This modification of 
normal procedure is to be avoided if at all possible in order to con- 
trol the conditions under which respondents are asked to provide infor- 
mation . 

£-33 Explaining the Questionnaires to Respondents 

Forms 03 and 04 each include appropriate sets of instructions. 

It is the team leader's responsibility to review these instructions in 
detail with site personnel and to answer any questions which might 
arise as a result of that review. (By team leader, in this case, is 
meant the leader of the initial site visit and/or the data collection 
team.) In providing a review of the instructions emphasis must be 
given to the necessity for responding to every item contained in the 
questionnaire and to the fact that responses will be kept confidential. 
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As a final step in explaining the questionnaire, the team leader must 
work out with site personnel the procedures to be followed in cases of 
serious doubt about any item wliich may cause the respondent to be 
unable to respond to it. Generally this will consist of arriving at 
an understanding thau all quev'' tionnai res will be reviewed by the data 
collection team and, where omissions or questions are noted, that the 
involved respondents will be contacted further. Procedures for return 
of the questionnaires to site visitation personnel must be worked out 
at this time. By doing this, respondents can be assured that confiden- 
tiality of the responses will not be violated by other site personnel 
informally "reviewing" the responses. 

E-34 Collecting the Questionnaires from Respondents 

Assuming optimal operation of the methodology, the leader of the 
initial site visit has established cooperation with the responsible 
head of the site for return of all questionnaires. To assu-»"e main- 
taining the confidentiality of respondent data, questionnaires must 
be returned directly to the team leader or a member of the data col- 
lection team. Procedurclly, the questionnaires are returned in per- 
son at the time of an interview. At that time the data collection 
team member receiving the questionnaire must confer with the respond- 
ent to complete Item 47 of Questionnaire Form 03. Using Category 
Set X (Project Role; see Appendix X) as the frame of reference, agree- 
ment must be reached as to which of the roles listed in Category 
Set X best describes the respondent's role in the project. The code 
number for that role is r^^orded by tne data collection team member 
in the box provided for Item 47 in the questionnaire. 

At this time also, the adequacy of the completion of the ques- 
tionnaires must be examined. In the presence of the respondent , the 
team member collecting the questionnaire looks quickly at each item 
to assure that they have been responded to. Any notations on the 
forms, or any questions raised by the respondent, are clarified and 
any necessary changes in the responses are made at this time. The 
team member makes special note of any problems encountered by respond- 
ent so that subsequent explanations or revisions of the questionnaires 
can reduce the sources of such ambiguities. He then closes out the 
ques tionnaire phase of his data collection effort. 



Activity Block 7: Onsite Data Reduction 



Output Index ; 

E-35 Recapping (component, production) 

E-36 Finalizing the Output Map (component, production) 
E-37 Contextual Debriefing (component , production) 

Output Map ; 



E-35 



E-36 



E-37 I 
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rhe first data reduction activities are performed by the interview 
team while still onsite. These activities include recapitulating 
interview data (recapping), finalizing the output map, and contextual 
debriefing. The data collection team is the primary source of first- 
hand knowledge about the site. During the team’s visit they interview, 
discuss, and observe, noting (physically or mentally) many impressions 
relating to the overall context and working conditions surrounding the 
project/site, and they particularly form impressions of the meanings of 
statements made by interviewees. These latter impressions tend to be 
lost if not used quickly in recapping the recorded interview data. The 
recorded tapes preserve only the ’’audio.” The scene is preserved in 
the interviewer’s memory. For this reason, if recapping were done by 
someone other than the interviewer, the benefit of the interviewer’s 
affect in the site context would be lost. 

Further, reviewing the data (by recapping) enables the interviewer 
to detect omissions in the information gained from an interviewee. If 
the recapping does reveal such a gap, either a second interview or a 
few questions could be directed to the interviewee to complete the data 
while the team is still ons^ite. If such gaps are not detected until the 
team has departed from the site, the cost of filling them is high, 
whether by telephone calls or a return visit- Especially costly is the 
loss of the frame of reference and rapport Wwich was established during 
the team visit. For these reasons, it is most desirable for the recap- 
ping to be accomplished by the interviewer as quickly after the inter- 
view as possible, and in any event, prior to the team’s departure from 
the site. 

E-35 Recapping 

The term ’’recapping” as used here, indicates the first step in 
data reduction. Basically, it is the identification and recording of 
data in written form. The term is used in relation to: 

1. Output identification (Form 05 -A) ; 

2. Transfer of output identification numbers to output data 
sheets vForm 05-B) , coupled with some limited coding; 

3. Transfer (and/or translation) of tape recorded interview data 
into statements of standards (Form 06), tasks (Form (07), and 
enablers (knowledges, skills, and sensitivities — Form 08); and 

4. Transfer (and translation) of tape recorder recommendations 
for training into summarized statements (Worksheet 13). 

Recapping consumes a major portion of the data collection team’s 
onsite time. Although considerable variation will be noted for indi- 
vidual interviewers, the average recapping time is approximately 2.5 
times that spent in interviewing. Thus, if an interviewer conducted 
two interviews per day, each of approximately 2 hours duration, the 
approximate time spent in recapping will be 10 hours, for a total time 
of 14 hours spent in interviewing and recapping. 

Recapping centers around listening to tape recordings. In order 
to reduce interruptions, preserve the confidentiality of the data, and 
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eliminate interference with others, it is strongly recommended that the 
recapping occur in privacy. A supply of blank Forms 05-A, 05-B, 06, 07, 
08, and Worksheet 13 are required. 

In recapping the interview from audio tape to a written form, the 
following procedures should be followed: 

1 . The first step in recapping is output identification . Each 
output which served as a point or focus of the interview must 
be identified by a title and number. If the Form 05-A has 
been comple ted and finalized, the output is located on the 
Form 05-A and its identity number obtained therefrom. However, 
if the Form 05-A has not at this time been finalized, the out- 
put title should be written on a blank Form 05-A. In this 
case, coordination must be effected through the team leader 

to insure that a single identity number is matched with an 
output title and is not used by different interviewers to 
identify different outputs . 

The identifying output title should be worded in a descrip- 
tive fashion and agreed upon between interviewer and inter- 
viewee. The agreed upon wording for the first output inter- 
rogated around will be found in the early part of the first 
tape recording of the interview. The project identity number 
(a three-digit number assigned prior to the site visit), the 
output identity number , and the output title must be con- 
firmed on Form 05-A. The project identity number and output 
identity number are then entered in the appropriate spaces on 
a blank Form 0‘5-B. 

2. The two identification numbers (project and output) are then 
entered in the appropriate space on one blank copy each of 
Forms 06, C7, 08, and Worksheet 13. (Forms 06, 07, and 08 
may be printed on colored paper for ease of handling; e.g., 
06-red, 07-green, and 08-yellow. ) 

3. Each interviewer is identified by a two-digit number. Enter 
the interviewer number on each of the Forms 06, 07, and 08, 
and on Worksheet 13. 

4. Upon determination by the team and team leader of the outputs 
to be interrogated around and the site personnel to be inter- 
viewed, a two-digit identity number is assigned to each inter- 
viewee. This interviewee number is entered in appropriate 
space on Form 05-B. (Use the left hand position in this 
column. Additional spaces are provided for later use since 

it is possible that more than one interviewee may be ques- 
tioned in relation to any one output . ) Enter this same inter- 
viewee number in the appropriate space on Forms 06, 07, 08, 
and Worksheet 13. 

5. As an advisable (but not essential) identification step. 

Forms 06, 07, and 08 can be headed by the output title in any 
available blank area at the top of the forms. This wording 
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can prove to be a useful crosscheck of output identifica- 
tion, and a reminder of the output (on each sheet) for the 
coding resolution team. The result of this identification 
step is to insure that all forms pertinent to one output con- 
tain matching identity information and can at any time be 
retrieved from separate files and considered as an entity. 

The remaining spaces on the Forms 05-B, 06, 07, and 08 are 
left blank at this tiroe . 

6. The tape recording of the interview is now closely listened 
to, to detect the various s tatements of "standards , " "tasks , " 
and "enablers’* as made by the interviewee. (Since interviews, 
though centered on a single output, do not always proceed in 

a well ordered manner, it is necessary to have all three forms 
readily at hand.) The recapper must, in effect, relive the 
interview and regain the unders tending of the interviewee ’s 
words that he had during the interview. Frequently, in re- 
sponse to questioning in relation to standards, an inter- 
viewee may qualify the standard by mentioning enablers. For 
example, the interviewee could say, "Because 1 have been a 
newspaper reporter, I have a feeling for what will be accept- 
able to ‘^or will antagonize a reader. Therefore, I read the 
drafts of reports so that I feel certain that we will not 
create a poor impression." This statement indicates a standard 
that the interviewee personally applies to the generation of 
the output. However, in addition to the standard, he hints at 
a skill in newspaper-style writing and a sensitivity to reader 
reaction. Although the statement was elicited as a standard, 
the skill and sensitivity can and should also be identified and 
recorded. 

In the example given, the standard was worded negatively (e.g., 
"I feel certain that we will not create a poor impression."). 
The term "standard," as used in this methodology, is of the 
degree of acceptability or of excellence. Also, the wording 
of the standard statement to be written on Form 06 should be 
as nearly in the interviewee's own words as possible. The re- 
capper, in attempting to meet these criteria, could record on 
first line of Form 06 "I feel certain that we will create a 
good impression . " 

7. Since, in the above example, the interviewee indicated that he 
applies this standard himself, the numeral 1 would be entered 
in column K beside the written standard. If, instead, the 
standard is applied by one or more persons other than the 
interviewee, the numeral 2 is used in column K. And, if the 
standard is applied by the interviewee working in conjunction 
with one or more others as a team, the numeral 3 is used. 

8. If, as in the example cited, a task or enabler is detected or 
alluded to, a tentative note can be made on the appropriate 
form so that the noted item can be confirmed as being present 
when that particular portion of the data is being considered. 

9. If additional pages are required for recapping standards, 
accurate identity information must be placed on the additional 

i ^ 
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sheets. Lf no standards were elicited in relation to an out-' 
put, the Form 06 must indicate this by enterlrg "no information 
collected under this heading." However, such a no»-response 
sheet should be retained. While recapping, all pages should 
be kept carefully in order for later page numbering. 

10. When all standards related to a particular output have been 
recorded (or believed to be written), Form 07 may be used to 
record tasks done in relation to that same output . 

In content, a "task" is generally said to describe a worker 
accivity. It is a unit of activity that deals with the methods 
and procedures and techniques by which outputs are produced. 
Tasks are discrete units of work performed by an individual 
or individuals where each usually has a definite beginning 
and ending within a limited period of time. They are not 
so finite as to constitute a single homogeneous action of the 
worker, but are compos i te s tatements describing meaningful 
units of work performed. 

The task statement is composed of three basic elements: 

(ai) a specific verb describing an action; 

(_b) a brief identification of what is acted upon; and 
(_c) whatever qualifying phrases are needed to clearly dis" 
tinguish that task from related or similar activities, 
or to limit and define the scope of concern. 



For example: 
"Counsel 

-E 

(action) 

"Compute 

-f- 

( act ion) 



staff personnel 
(item acted upon) 
product moment correlations 
(item acted upon) 



-f- 



-f- 



on career advancement" 
(qualifier) 
on a desk calculator" 
(qualifier) 



Task actions basically are of four gross types: 

(_a) Information gathering activities . These are tasks performed 
primarily to obtain information about a job situation or 
the degree to which progress is made in the attainment of 
the job goals and standards. Typical verbs are: observe, 

review, receive , moni tor, ob tain , listen, e tc . 

(^) Determinations . These are tasks performed in which judg- 
ments are made about the existence, adequacy, or nature 
of deficiencies of a job situation. Typical verbs arc: 
decide , determine, judge , es timate, etc . 

(^) Controlling activities . These are tasks performed to 

correct, maintain, operate, or report directly on a job 
situation. They directly seek to chaage a state of affairs 
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1.0 j)rovl(li» pr<JK*^caa toward Uic /ii t /i I niiifii i ol Job 
;ui(j .‘standards. Typical verba are: instruct, assign, 

rnforin, counsel, submit, phone, compute, recommend, 
plan • etc . 

(jd) Generating activities > These are tasks performed that 
result in producing original components or elements 
of outputs. Typical verbs are: write, prepare, gener- 

ate, create, make , build, draw, etc. 

Task statements should not contain action words that are not 
sufficiently specific. The use of such words results in task 
statements that do not meet the requirements for finite and 
meaningful work units. Typical of such verbs are: supervise, 

insure, oversee, manage, etc. Such words should be used as 
cues to identify a need for more interview probing. 

Items acted upon basically are of three gross types : 

(a) Things (equipment, materials, measuring instruments, etc.) 
(_b) Data (concepts, analysis, reports, tabulations, etc.) 

{c) People (peers, supervisors, advisor, subjects, self, etc.) 

Qualifiers may apply to either the action or the item acted 
upon . 

Examples of qualifiers applying to the action are: 

"Conduct weekly meetings with division directors ." 

"Compile reading program data into meaningful format ." 

"Assist district personnel in establishing and phrasing 
objectives . " 

"Interact with district personnel in such a way as to give the 
district research associate credibility . " 

Examples of qualifiers applying to the item acted upon are: 

"Identify relationships between the objectives of various 
projects . " 

"Develop an evaluation proposal." 

"Project requirements for the collection of preliminary data ." 
"Prepare a statement of project definition . " 

11. Frequently interviewees will provide task statements that 
describe work performed by others than themselves. This char- 
acteristic is indicated on Form 07 in the P column for each 
task. The numeral 1 is used to indicate that the task was per- 
formed by the interviewee, "2" is used to show tasks performed 
by other persdn(s), and "3" identifies those tasks performed 

by the Interviewee working with one or more other people as a 
team# Columns N, 0, Q, and R are left blank during recapping. 

12. If (as is frequently the case) more than one sheet (Form 07) 
is required, the identity information must be entered at the 
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top of the additional sheets. As with the handling of the 
standards sheets, their order should be carefully maintained 
pending page numbering . 

13. Enablers are considered to be one of three types! knowledges, 

skills, or sensitivities. Enablers may eithei be prerequisite 
to working on the subject output, or be acquired while doing 
the work. However, they are statements of knowledges, skills, 
and/or sens itivi ties that are considered by the interviewee as 
essential to the successful accomplishment of the output. 
Knowledges may either be stated as ‘'Knowledge of 

or be written exactly as stated by the interviewee. However, 
a statement of a knowledge should be identified by placing 
the numeral 1 in the S column on Form 08 preceding the state- 
ment , 

14. Skill statements may either be stated as “Skill in “ 

or “Ability to , “ or be stated exactly in the inter- 

viewee's own words. However worded, skill statements are pre- 
ceded by the numeral 2 in the S column, 

15. Sensitivity statements may either be stated as “Sensitivity to 

,“ or be written in the interviewee's own words. This 

category includes statements of requisite personality charac- 
teristics, Each sensitivity statement is preceded by the 
numeral 3 in the S column of Form 08. There is no necessity 
for grouping all knowledges, skills, and sensitivities together 
as sets , The numeral in the S column will allow for later com- 
puter sorting, 

16. Each enabler should be identified as being a requisite of: 
the individual being interviewed (//I) ; other person(s) (//2) ; 
or a team requisite (//3), This numeral is placed in column T 
on Form 08 preceding each enabler statement. Columns U and V 
are left blank during recapping, 

17. Additional sheets of Form 08 may be used as necessary making 
sure these additional pages are properly identified. As with 
previous foms, the order of pages must be rigidly maintained 
pending page numbering at the completion of recapping, 

18. After extracting the tape recorded information related to one 
output, determination of its structure must be made (or con- 
firmed). It is considered to be either a “product," “event," 
or a “condition" (see definitions in the Glossary). This de- 
cision is indicated in the box marked A on Forms 05-B, 06, 07, 
and 08. The numeral "1" placed in the A box indicates a product, 
“2" indicates an event, and "3" signifies a condition, 

19. The decision as to whether an output is either a product, an 
event, or a condition may have been made prior to the interview. 
However, the data gathered describes the output in great detail 
and may tend to confirm or refute a previously determined class. 
If a change is indicated, this information should be brought to 
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the attention of all team personnel to insure that the change 
is reflected in all documentation (e.g,, output map, profile, 
etc,). If no decision can be reached at this time, the A code 
may be left blank, during recapping but noted for referral to 
the coding resolution team, 

20, The relationship of an output to other project outputs is 
determined by the interview team in finalizing the output map 
(see the seotion in this chapter on finalizing the outpp.t map) . 
One aspect of this relationship is the level" the output 
holds in the overall project. The output is considered to be 
either: 

P'ocal “ a classification given an output having contractual 
relationship to the outcomes of the efforts of a 
project (identified by the numeral 1), 

Component - a classification given an output having an integral 
ingredient relationship to and which is not an 
instance of a focal output (identified by the 
numeral 2), or 

Facilitating - a classification given an output having an 

assisting relationship to and which is not an 
instance of a component or focal output 
(identified by the numeral 3) ,7 

The numeral indicating the level of the output is placed in 
the box marked B on Forms 05-B, 06, 07, and 08, 

21, One other item of information in relation to an output must be 

recorded: its completion status. Using the six-possibility 

code (see Appendix B2, set W) , enter the appropriate number in 
column W on the Form 05-B, This completion number appears on 
no other form. 

22, To this point, the recapping has dealt with a single output. 

That output has been identified by number, has been related to 
an interviewee and interviewer, has been matched with stand- 
ards, tasks, knowledges, skills, and sensitivities, and has 
been coded to show its structure, level, and stage of comple- 
tion, This entire set of recapping activities must next be 
done for the second and third outputs until all output data 

from an interview have been extracted from the tapes and recorded 
on the proper forms, 

23, When this activity is completed. Forms 06, 07, and 08 are then 

numbered using the upper right hand corner (page of pages). 



It is especially critical that the reader refer to Appendix 5 
for a discussion of the decision rules pertaining to these classifications. 
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Number all of these forms sequentially and insure that they 
are in proper order (06, 07, 08 for first output; 06, 07, 08 
for second output; etc.)* This set of forms represents the 
output specific data gained from one interviewee and should 
be packaged (folder or envelope) and labeled for subsequent 
delivery to the Coordinator of Data Inputs (see //17 within 
the section of this chapter on coordination of data input and 
storage). 

24. One more category of information must be extracted from the 
tapes . This information deals with the interviewee 's sugges- 
tions for training people to do the kind of work he has de^ 
scribed in producing his outputs. His suggestions should be 
summarized and written in narrative form in the left hand 
column of Worksheet 13. The right hand column is used to 
note the output, task, or enabler to which the training sug- 
gestion relates. Where training has been instituted as an 
integral part of the project , some comments may indicate 
training ideas found to be effective. Additional pages of 
Worksheet 13 may be used if required — ^but if used, additional 
pages must be properly identified. These pages are considered 
a separate document, paginated as a separate package, and 
placed with the Forms 06, 07, and 08 for delivery, with all 
tapes, to the Coordinator of Data Inputs. 

25. Forms 05-A and 05-B are composite forms for the entire site/ 
project, and will be completed while recapping later inter- 
views. Upon completion of the interview team’s site visit, 
one complete and final 05-A and 05-B representing all out- 
puts detected for that project will be delivered to the 
Coordinator of Data Inputs. 

E-36 Finalizing the Output Map 

Development of an operative output map is one of the essential 
elements of this methodology. To replicate the generation of outputs 
of the type for which a project is responsible, an important factor is 
understanding the dependency of the various component and facilitating 
elements with which the focal or contractual outputs are built. The 
map presents this understanding through a visual, schematic represen- 
tation of output dependencies, rather than a time-sequenced process 
with unstated relevance . 

Map finalization is a task of the interviewing team, with the team 
leader being responsible. Since all inputs necessairy for its produc- 
tion must be available before the team can finalize this visual repre- 
sentation, it is important that the team leader secure the necessary 
input before the team leaves the site. Finalization of the map will 
reflect these inputs from the site in terms of the dependencies discov- 
ered from interviewing around the various outputs . Initial versions of 
the map will also help in ordering the final visualization of '^utput 
dependencies . 



'[‘he following procedures should be observed in finalizing the out- 
put map : 

1. Upon completion of Form 05-A, the identity numbers of outpvi.ts 
are transferred to Form 05-B and certain codings are entered 
for each output. These entries specify the structure of 
output (product , event , or condition) , and the output leve 1 
(focal, component , or facilitating) . 

2. With the team leader moderating (preferably with a large wall 
chart), the team discusses and confirms that those outputs 
listed as "focal" on Form 05-B actually are the contracted-for 
project outputs . When consensus is reached , the focal outputs 
(or their identity numbers) are placed horizontally on the wall 
chart at the top of the page . 

3. Next, each component and/or facilitating output is considered 
in the project’s context and related to one of the focal out- 
puts by connecting lines. Each is marked on the chart using 
dependency relationship as the only factor. Time relationship 
is not considered. 

4. Many facilitating outputs make possible the accomplishment of 
more than one output. This multiple relationship is either 
shown by the use of brackets (see Figure 3), or reentered in 
another location, if that will more clearly convey the rela- 
tionship. In somB cases, certain outputs facilitate all other 
project outputs (e.g., staff welfare). Here too, brackets 

are used to indicate this multiple dependency. 

5. With every output detected, consensus is reached within the 
team as to its dependency relationships . Of course , the under- 
standing of an interviewer about the outputs upon which he 
focused interviews is of primary concern. All pooled knowledge 
of the team about the outputs identified and the proj ect 's con- 
text are brought to bear on the finalization of the output map. 

6. From this wall chart, then, a tentative, final version of the 
output map is drawn. Upon the team’s return home, it is 
approved by the Coordinator of Data Collection (or redone 
until it is approved), and delivered to the Coordinator of 
Data Inputs by the team leader. 

E-37 Contextual Debrie fing 

This is an activity during the site visit that serves several pur- 
poses. During interviewing, an interviewer may make contextual notes. 
This can either be about something that is contextual to the whole ac- 
tivity of the project, or it can be about something contextual merely 
to the output being interviewed around, in which case it may provide 
subject for further discussion among the team, and/or for questioning 
of interviewees . 




i ^ 



46 



Such contextual notetaking can serve as a trigger for complete 
contextual observations after the day*s interviewing is completed. The 
interviewer may use his tape recorder to make a permanent, verbal 
record of his observations at greater length than is feasible in the 
writing of notes. Such contextual debriefing serves as a personal, 
individual facilitator or memory aid of the context in which the proj- 
ect operates. 

Another form of contextual debriefing is one in which the entire 
team engages, sometime after the end of a day^s Interviewing activities. 
This debriefing time provides an opportunity to share comments regard- 
ing the site. Its purpose will be sharpened by the conscious effort to 
modify or finalize the output map. Directions for the conduct of such 
a debriefing session cannot be given because (^) every project visit 
will have its own variation as to what is needed, and (M time is of 
the essence, as explained earlier. It is helpful for the profile writer 
if this session is tape recorded. A sensitive team leader can economize 
playback time for the writer^s later review of the tape by recording 
only when something is being said that he discerns as critical to 
"remember." 

Prior to the debriefing session, it may be well to establish pri- 
orities for what it should accomplish, e.g., to clarify a part of the 
output map, to address some specific problems, or to discuss unexpected 
features uncovered during the day^s interviewing, etc. This can some- 
times be done informally over dinner. It is difficult to make hard 
schedules, because every project is likely to have its own scheduling 
problems, both for individual interviewers and for the team as a whole. 
An alert team leader with a cooperative team can manage to insure that 
each team member*s time is optimized during the site visit. 

While part of the debriefing purpose is to provide a basis for 
revising strategy as may prove advisable during the visit, a second 
part is to get on record some of the contextual observations perceived 
by all of the team that otherwise might get overlooked due to simple 
"information overload" from seeing and listening to so much during the 
time onsite. 

In addition to daily team discussions of contextual information, 
a final team debriefing should be performed and recorded either while 
the team is onsite or during the return trip. Performed after all 
interviewing is complete, this final discussion of contextual informa- 
tion provides a summary of the team's impressions of v.he site. Sharp- 
ened by numerous interviews , personal contacts and frequent obse rva- 
tions over a period of time, the data collection team's knowledge at 
this time should be at its peak. The resulting tape recording should 
be a most useful tool to the profile writer. 

The final contextual debriefing may be either formal or informal 
as the team leaaer desires . Howeve r, each team member should be given 
an opportunity to present his views for the team's concurrence or 
disagreement . 
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Activity Block 8; Closing Out a Site Visit 



Output Index : 

£-38 Completeness Check (component , management) 
£-39 Final Meeting (facilitating, management) 



Output Nap : 



r E-3^ I 'e-39 

E-38 Completeness Check 

1. Worksheets 17 and 18 may be used to insure completeness of 
data gathered. 

2. Check that all Forms 02, 03, and OA (questionnaires) have 
been received. 

3. Check that all Forms 02, 03, and 04 have been properly 
completed. 

4. Check that all recapping has been completed. 

5. Check that output map is complete in final draft form. 

6. Check that all documentation (supplied by site) has been 
received and packaged . 

7 . Check that return transportation arrangements are 
complet'id. 

£-39 Final Meetin g 

The team leader should meet with the site director prior to the 
team^s departure. In some cases the director may wish to have a final 
meeting with the tiutire interview team. Regardless of the form the 
meeting takes» the following items should be covered: 

(^) Assurance of confidentiality in data compilations. 

(_b) Assurance that a copy of the site profile will be sent for 
the director's approval prior to printing. 

(^) Assurance that the final results of the investigation will 
be made available. 

(^) Sincere thanks for the site's cooperation . 

Frequently, team leaders carr/ thext responsibilities beyond the 
site visit itself by sending follow-up letters. Many lasting friend- 
ships and mutually beneficial professional relationships have resulted 
from the team/site interaction . 

The team leader alone does not carry the responsibility for team/ 
site affect. The professional and ethical conduct of every team 
member is instrumental in leaving the site with a positive affect. 
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Chapter 2 



DATA MANAGEMENT ACTIVITIES F0LL0W1N( SITE VISITATION 



Data management activities occur whenever there is any interaction 
between the investigating agency and the site being investigated. Some 
of these activities arc of such a ''common sense" nature that they have 
only been mentioned in passing in Chapter 1. A detaileti set of activi- 
ties that might be viewed as data management was included in Chapter 1 
(Activity Block 7: Onsite Data Reduc*ion), When the data collection 

team returns from a site visit, a major data management effort takes 
place. It is this set of procedures (indicated in Figure 2 by Activity 
Blocks 9 through 18) that this chapter addresses . 

This chapter will "walk" the potential user of this methodology 
sequentially through the major data management activities to the gener-' 
ation of tables pertinent to a single case (site) and the generation of 
cross-case analyses and displays. The following major headings will be 
used. They correspond to the Activity Blocks displayed in Figure 2; 



Activity Block 9 
Activity Block 10 
Activity Block 11 
Activity Block 12 
Activity Block 13 
Activity Block. lA 



Activity Block 15 
Activity Block 16 
Activity Block 17 
Activity Block 18 



Coding Questionnaire Data 
Coding Outputs 

Coding Work Requirements Data 
Category Set Development 
Checking Coding Consistency 
Computer Storage of Data in Prototype 
Data Bank 

CTiecking of Data Reliability 
Analyzing and Displaying Data Across Cases 
Computer Output of Profile Tables 
Coordination of Data Input and Storage 



Activity Block 9: Coding Questionnaire Data. 

i 



Output Index ; 

E-AO Coding Ques tionnaire Data (component , production) 
Output Map : 



E-AO 



E-AO Coding Ques tionnaire Data 

By the time that Forms 02, 03, and OA are ready to be prepared for 
data input, some coding has already been performed. Th.is coding con- 
cerns such things as the assignment of project identification numbers, 
personnel identification, region numbers, the determination of project 
focus, project complexity, stage of completion, soj or. ’cry foci, and 
project role. 
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This coding involves the use of some judgment and therefore must 
[)v ]j»*rr(jrmed by someone familiar with the project in question. This 
responsibility is usually assigned to the data collection team leader. 

The coding which remains to be done consists of determining an 
appropriate code number from a list of code numbers for each item or 
question on Forms 02 and 03. The codes require no judgment beyond 
that of recognizing which category within a variable has been indicated 
by the respondents. 

The codes to be utilized in coding Form 02 and 03 iteras are con- 
tained in Appendices 3 and 4. The appendices indicate which questions 
on Forms 02 and 03 require coding, what variable/level combination 
each question corresponds to, and what each code number represents. 

Form 04 information does not need to be coded as each respondent indi- 
cates by his numerical response which category of response he intended. 

After these codes are determined, the data are ready to be prepared 
for input to the cemputer. At this point a decision must be reached as 
to whether hand tallies or summaries of the open-ended questions or 
other data which are not utilized in the computer data-storage files 
are to be obtained. If the answer is in the affirmative, then an indi- 
vidual must be assigned to perform a hand summarization of the data. 



Activity Block 10: Coding Outputs. 



Output Index : 

P-41 Coded Outputs (component , production) 

E-42 Confirmation of Data Collection Team's Coding 
(facili tating, management) 

E-43 Output Coding (component, production) 

Output Map : 



(T-ai 







1 E-A2 


E-43 1 



P-41 Coded Outputs 

Coding, as performed by the Coding Resolution Team, is a time con- 
suming job performed by at least a twe-man team. For planning purposes, 
an average time could be considered to be 6 working days (for a two-man 
coding team), to code the data gathered by a three-man data collection 
team during a 1 week site visit. This time estimate is subject to 
many variables, but assumes that an average day of coding would consist 
of approximately 5 hours of coding time (10 man-hours). The coding 
team will develop proficiency with practice and familiarity with the 
category sets. Other variables include the proficiency of the data 
collectors, the availability of the interviewers, the quality of the 
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recapping, etc. Approxluiately 50% of coding time is consumed by output 
coding. All output codings are entered on Form 05-B. 

E--42 Confirmation s, f Data C o llection Team^s Coding 

Though the data collect ior. t-cams are charged with coding columns 
, B, and W, all entries made Ky them must be checked and confirmed . 

1. The first output coding step is to verify the project identity 
number and page numbering, and to compare the output map with 
Form 05-A to insure that the output listed on the 05-A also 
appears cu the output map end is pro*».^rly identified on both 
documencs by the same ident.' f icat ion number. 

2. Next, all output titles on the 05-A aeC read to insure both 

legibili ty and understandability . If any are : eking in these 
respects , the appro,) r data collection team member (inter- 

viewer or team leader, } is called in and a mutually acc'^ptablc 
descriptive output title is agreed upon and recorded on the 
05-A. Form 05-A lists all outputs identified relating to a 
project/site . 

3. The 05-A list is next compared with the data in 05-B to j "isure 
that page numbering , projec t identity number, and output 
identity numbers match between the two forms. (Identity numbers 
are on both forms - titles are on 05-A only ) 

4. On Form 05-B, those outputs around which interviews were con- 
ducted are indicated by interviewee numbers appearing in one 
or mhre of the center columns. For convenience and rapid 
reft, fencing , these outputs can be asterisked ip the left mar- 
gin of both Forms 05-A and 05-B. 

5. All spaces on 05-B, other than interviewee numbers, are coded 
for every output identified, whether it is interviewed around 
or not. Column A coding identifies the structure of output: 

//l indicates a product, //2 indicates an event, and //3 indi- 
cates a condition. (Set A - Class)^ 

6. In similar fashion, column B (Set B - Level) and column W 
(Set W - Completion) are checked and confirmed. If items are 
missing or not understandable, then they are resolved in con- 
ference with data collection team members. 

* 

At this point in the output cooing, all project outputs identified 
for the site have been numerically described as follows; 

1 by a discrete identity number (output identification) 

2 by structure (a product, an event, or a condition) (A) 

. 3 by level (focal , component , or facilitating) (B) 

4 by stage of completion (completed over a year ago, completed 
3-12 months^ agOj completed within last 3 months, currently 
in progress, not yet started, or ongoing) (W) 



All category sots referenced in:, this volume are in Appendix 4 . 
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'j Output Coding 

Columns CDE, FGt'L, Y, Z, and ’’a" remain to be coded. Column Y 
(Set Y - Project Focus) is an entry that is identical for all outputs 
of a site and represents the project focus (e.g., a research project, 
a development project, a diffusion project, or an evaluation project) . 
This designation was tentatively determined for each project during 
site selection. Hawever, it is .sometimes changed subsequently. Either 
the project titla or itt description may have conveyed the impression 
that its activity related principally t. one focus, when its actual 
focus, within the definitions used by this project, proved to be some- 
tiling else. This could happen because a different definition of re- 
search, development, diffusion, or evaluation was used by the people 
who wrote the proposal for the project being visited. More likely, it 
was something in the wording that implied a focus based on the defin^*- 
t: ons used by this project. In any event, the existence of a different 
principal focus from thaw originally identified may be confirmed. 

Usually some intimation of a difference in definitions, at least 
as a possibility, will occur during the initial site visit, but may 
need to be confirmed during the team visit. Or, it could be that the 
focus of a number of focal or other outputs consistently indicates that 
the majority of the effort within the project supports a focus differ- 
ent from that previously identified. For example, a preponderance of 
implementation outputs in a project thought to be a research project 
could require that the project designation be reconsidered. 

1. Column C (Set C - Character) represents the character of the 
output icse.lf. Its determination is the responsibility of the 
coding resolution team, and may involve the use of decision 
rules . ^ 

2. Column Z (Set Z) indicates the character of the focal output 
to which a component or facilitation output contributes. In 
column Z all focal outputs are coded as being focused on them- 
selves, and outputs supporting the project generally are coded 
with the focus of the project. 

3. Column "a” reflects the function served by the output. There 

are three classifications used to designate functions: 1 = 

Policy, 2 = Management, and 3 = Production (Set '*a" - Function) 
Again, as in most of the categories, decision rules have been 
developed to ass is t in this classification . 

4. The two remaining columns, one labeled DE and the other FGHI, 
are category sets that cluster outputs of similar nature. 
Originally, the FGHI set (primary category) was expected to be 
a complete listing of all outputs. However, it. was soon 



^ See Appendix S. 
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obvioub that different: sites produced nearly identirai out- 
puts, and were coded with the same FGHl number (hencr, a pri- 
mary categorization). As the FGHI list of primary categories 
grew to unwieldy size, primary categories that had commonali- 
ties were clustered (hence, DE - Cluster Categories). 

In operational usage, the DE coding is deLtrmined first as a 
gross indexing which in turn leads more quic < ly to determining 
an appropriate FGHl code. Both DE and FGHl sets are open 
ended and can be expanded as the incoming data warrant. The 
coding resolution team either selects existing DE and FGHL 
categories fitting each output or uses the output to establish 
new categories j.f a fit with existing categories cannot be 
made . 

5. To clarify the interrelationship between category sets A, DE 
and FGHl , the DE and FGHl categories are ordered uniquely 
under the three A codes. Tliat is to say, any specific ele- 
ment of the DE or the FGHl se t can only appear \n one of the 
three A codes, as a product, an event, or a condition. The 
DE set then designates a broad delineation ot the item within 
one of those sets, largely by the purpose the output serves, 
in relation to other outputs with which it is dependency 
related. 

This should not be confused with the designator applied in the 
B set, which project-specifically states whether a given out- 
put is fqcal, component, or facilitating. 



- i 

Act ivi ty Block 11 : Coding Work Requirements Data . 



Output Index : 

E-44 Transfer of Output Codes to Recap Sheets (component, 
^ management) 

E-45 Coding Standards (component, production) 

E-46 Coding Tasks (compo at, production) 

E-47 Coding Enablers ( onent , production) 

Output Map : 




E-44 Transfer of Outp i ^*: Codes to Recap Sheets 

Within a package of recap sheets (representing the data gathered 
from one interviewee, and numbered consecutively) are sets of Forms 06, 
07, and 08 for each output. Taking one set at a time, the appropriate 
output code numbers are transferred from Form 05-B to the recap sheets. 



li/n h (wlu!t)u*r Form 00, 07, or Oti) rol/iLi'd lo oiu* oiilpuL Is m,irkc*d 

1 flciiL i c a 1 I y , Thin action is taken on each set of recap sheols mw. I 1 a 
package Lb properly marked. Kach of the following packages La handled 
Ln the same way until every recap sheet for the entire site Is properly 
marked. This transferral action, though clerical in nature, is per- 
formed by the code resolution team. Meticulous attention to accuracy at 
this point is critical. Subsequent insertion of data into the computer 
is made from Form 05-B and separately m'^de from the recap sheets. Later 
analyses and sorts retrieve from both sets using the code numbers com- 
mon to both. ITie two sets of code numbers must be identica l. 

At this point in the ceding process only one column labeled 
’’Page No.** remains blank on all recap forms. This column is used by 
the data analysis team when sorting pages for computer insertion. Note 
that these packages of forms ultimately get two sets of page numbers, 

one (page of ) which serves coding resolution purposes, the other 

for computer insertion purposes. (See step 27 in the section of this 
chapter on coordination of data inputs and storage.) 

E-45 Coding Standaras 

Standards are considered in relation to their associated outputs 
in the next coding step. If the standard is used to judge the accepta- 
bility of the output itself, category set J-1 is inspected to seek an 
appropriate category. If the standard is used to judge the process or 
operation leading to the output, an appropriate category should appear 
in set J-2. The two coders, working separately make their selections 
of categories ai<d then compare choices. Concurrence must be reached. 
Lack of concurrence may indicate that a new category is required. 
Assuming that the coders agree upon a category, that category number 
is placed in the column preceding the statements (J-LM) . The K 
column has already been filled in by the recapper. (If not, the coders 
must determine the self-others status by discussion with the recapper.) 
This procedure is repeated for each standard statement rel-^ted to the 
first output as recapped from the first interview. 

If a statement is found to be inadequate, the person who wrote 
it is called in or contacted. For example, a standard statement may 
read, ’’Whether or not ....*' This is an inappropriate wording for a 
standard. The wording needs to be changed so that what appears is a 
standard of acco.puability . Failure to meet the acceptability standard 
automatically implies nonacceptabili ty . Another example s tat erne nt 
might be, ’’The language is appropriate to the target audience.** This 
form will usually be considered insufficiently specific to use as a 
working standard. In what way does tha language have to be appro- 
priate to the target audience? It could be that the level is such that 
the audience can uixderstand it. It could be that it avoids words or 
expressions that might be offensive to the target audience . More posi- 
tively, it could imply that language is used that excites the targe t 
audience’s interest in the subject matter. 

If the interviewer who rt:capped such a statement as this has no 
further information to elaborate, this is a good occasion to sensitize 
him to the need for probing to insure that his data are gathered at an 
appropriate level. Hov/ever, the interviewer may have sufficient data 
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on his interview tape to determine what is needed and may be able to 
more clearly and definitively recap the inadequate statement. 

E-46 Coding Tasks 

The basic procedure used with standards is repeated for tasks, but 
tasks have their own peculiarities. One of the tendencies in writing 
tasks has already been mentioned in the instructions for recapping: 
the use of statements (particularly clued by the use of certain verbs) 
that are too gross, such as "oversee the work of subordinates in the 
department." Of what does overseeing consist? l^at did the interviewee 
dp that constituted overseeing? 

Another weakness often found in task description is that the word- 
ing does not adequately identify what work is performed. It may have 
been quite evident to the interviewer from the context of the inter- 
view. If not, he would have pursued the questioning for clarification. 
In writing down the response, recappers sometimes tend to omit what, 
at the moment of writing, seems "obvious," whether it be the verb (not 
very often), the item acted upon, or a necessary qualifier. This was 
discussed under "Recapping" (Chapter 1) at some length. 

Tasks have a two-level arrangement of category sets The NO code 
describes the first-level designation of the task. Within each NO code, 
there is a subset of codes with QR designators, unique to that set. 

Thus the QR designator "01" will have a different meaning in the NO set 
"15" from that in NO set "16" — or in fact in any otler NO set. Once 
concurrence is achieved, the coding resolution team enters the selected 
task category numbers in the appropriate column on Form 07. 

The P code, like the K code for standards and the T code for ena- 
blers, is one the recapper should have inserted to indicate whether 
the interviewee does the task himself (enter '"l"), if other (s) do it 
(enter "2"), or if the interviewee does it in collaboration with other(s) 
(enter "3"). If the recapper has not entered these codes, the proper 
entry may be obvious. For example, he may include the relevant pronoun, 
"I read what someone else has written," "We discussed the significance 
of the dc^ta," "They report to me what they have found," etc. Wherever 
the wording does not provide a reliable clue for this coding, the matter 
should be checked with the recapper. This procedure not only insures 
that this coding is accurate, it insures that it will be filled in by 
the recapper next time. 

E-47 Coding Enablers 



In coding enablers, the recapper should have filled in the J and 
K codes, to identify whether the statement represents a knowledge, 
skill, or sensitivity, and whether it represents such an enabler pos- 
sessed by the interviewee, or by someone else, or by <:.ll members of a 
team. The context can be very.;^*important in determining into which cat^-^- 
gory an enabler falls. 

The trained interviewer is in the est position to determine where 
the wording may leave the matter dubious, because he possesses the 
context of the interview In which the response v/n<; evf;krd. Tiie "le.id” 
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words are not always reliable* For example, an inCerviowee may say, 

’’I hrwe to know when it is appropriate f - raise a certain issue.” The 
word **know'' inunediately implies that he is going to describe a knowledge, 
bat tlie rest of his statement suggests that he is in fact talking about 
a sensitivity* 

It would be hoped, in the above example, that more precise infor- 
mation was obtained: "appropriate” could mean "relevant” or it could 

have other connot 9 tions . If the meaning was that of relevance, then 
the statement could be more representative of knowledge out of which 
relevance would be recognizable, or of the skill in making such a 
recognition. A good interviewer would probably have obtained responses 
to indicate all three (or some similar) enablers* 

If the J and K codes have not been provided by the recapper, the 
coding resolution team must either determine the proper codes from the 
statements themselves or by conferring with the recapper. 

Once an enabler statement has been confirmed as being a knowledge 
(1), a skill (2), or a sensitivity (3), the coding resolution team 
selects, from the appropriate list, the most nearly matching category 
or creates a nev^ category if none fits * Once concurrence is achieved, 
the code numbers are entered in the appropriate columns on Form 08* 

Each set of recap forms for each output is completely coded until 
the entire data from a site have been completed* New categories are 
added to the appropriate sets as the coding resolution team determines 
that data do not fit an existing category. In this way the coding 
category sets are de^J^eloped empirically * 

Upon completion of coding, each copy of each form used (05-A, 

05-B, 06, 07, 08) is stamped as approved by the coding resolution team 
and all data froia that site are returned to the Coordinator for Data 
Inputs tor further processing . 



Activity Block 12: Category Set Development 



Output Index : 

E-48 Empirical Expansion of Category S^ts (component, production) 
Output Map : 



E-48 



At any one point in time during the employment of this methodology 
category sets for coding data are in exis tence • ^ One primary sough t- 
af ter output of such employment is a group of category sets that mos t 



See Appendix 4 for current category sets. 
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accurately and completely describes the domain. To acli ieve the mos t 
accurate and complete descriptions possible, the category sets must 
be. applicable to any data collected. If they cannot be so applied 
(i.e., data do not appear to be classifiable), then the category set 
in current use is nut accurate and/or complete . Only by expanding the 
category set to make possib le its use in classifying those data can the 
category set be again considered adequate for the sample of data 
ob tained . 



E-48 Empirical Expansion of Category Sets 

In view of the above concept, the coding resolution team attempts 
to code all Incoming data (both output data and work requirement data) 
using the existing category sets and decision rules. ^ If part of those 
data cannot be classified within the existing categories, it indicates 
that the categories need expansion. 

The coding resolution team carefully considers each unclassif iable 
piece of data, discusses that item of data from all viewpoints, and 
only after serious and comprehensive study, creates a new classifica- 
tion to fit the data in question. 

(Virtually all of the existing category sets were de^^eloped or 
modified in this manner during the Oregon Studies) . 

Each data set is considered as '*open" to change (or as being 
currently complete in light of the data brought in from the '*real 

world*0 . 

As this methodology is employed in other contexts, the category 
sets will become increasingly stable and therefore more universal in 
their ability to describe the domain from which the data was drawn. 

In expanding the category sets, the coding resolution team must 
fully evaluate the consequences of their actions, Tliey must detect when 
they are trying to "force fit" data into a category , It this force 
fitting has been done in several cases, and then subsequently a more 
appropriate category is created, considerable recoding may have to be 
done to correct the previous force fitting. 

If the application of this methodology includes interfacing 
of the data with the Oregon Studies data bank^ any expan- 
sion or modification of the existing category sets and the 
decision rules applying to the use of new or modified cate- 
gories must be documented so that the interface may be ap- 
pro priat e ly accomp lished. 
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See Appendix 5. 
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Activity Block 13: Checking Coding Consistency 



O utput Index ? 

E-49 Coding Consistency Instrument Construction for the Data 
Collection Team (facilitating , management) 

E-50 Coding Consistency Instrument Construction for the 
Coding Resolution Team (facilitating, management) 

E-31 Coding Consistency Data Analysis (facilitating, management) 



Output Map : 




The value of any method for data collection is directly dependent 
upon the consistent reduction of raw data by the data coders. Consist- 
ent coding requires thorough training including discussions of decision 
rules, coders* biases, and systematic procedures to resolve coding prob- 
lems. For this aspect of the methodology the coding of interview derived 
data is done by a coding team of two or more people. 

E-49 Coding Consistency Instrument Construction for the Data Collec- 
tion Team 



A number of important decic"^ ons as to the coding of data must be 
made by the data collection team. This is primarily due to the context 
within which data are collected. As an example, determining whether an 
enabler is a knowledge, skill or sensitivity is heavily dependent upon 
the words utilized oy the interviewee in describing a particular enabler. 
In some instances the statements could not, in recapping, carry the 
essential message as to what class of enabler was represented. It is 
therefore necessary to require the interviewer to supply both a narra- 
tive statement and a code number. In addition each data collection 
team should mutually .determine both the level and the structure of each 
output. 

A test of reliability for the site visitation team consists of: 

1. Checking the consistency of coding for enabler statements 
to insure that the correct enabler (knowledge, skill , or 
sensitivity) category is used, 

2. Checking the coding of narrative output statements on the 
basis c f level, and 

3. Checking the coding of narrative output statements on the 
basis of structure . 

E-5Q Coding Consiste n cy Instrument Construction for the Coding Resolu - 
tion Team 



A major problem in the determination of code consistency is the 
provision of **ob jcctive** criterion against whicli a code may be judged. 
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The coding resolution coam has the responsibility for coding narrative 
statements accordin'- to Output, Standard, Task and Enabler category 
sets. The activitl s engaged in require that each narrative statement 
receive a code numt ^r from one of the appropriate category sets. Any 
Instrument or technique developed to check reliability should theieiorc 
require that each coder interpret narrative statements and determine 
either an Output, Staivlard, Task or Enabler code for each. 

The coding of such narrative statements is dependent upon a num- 
ber of factors which must be recognized if an accurate measure of 
reliability is to be achieved. In the coding of narrative statements 
utilizing the Output category set the coder should bo provided with: 

1 . The narrative statement 

2. An output map which includes the narrative statement 

3. The statement identification number 

A. The Level Code 

5 . The StrvcLi r e Code 

6 . The Stage of Completion Code 



An output map is essential because the specific coding is context 
dependent . The map provides the coder with some degree of insight into 
what each narrative statement may represent in relation to other outputs. 
Items 3-6 are supplied by the interviewer and recorded on Fovnn 05-B. 

The coding team is therefore not expected to provide these codes. 

The coding team will code each narrative statement in cerms of: 

A. A Primary Output Category number 

B. A Cluster Output Category number 

C. The Output Function 

D. The Output Character 

In coding Work Requirement data the coding resolution team requires 
the following information: 

1. The codes determined above and recorded on Form 05-B 

2. The narrative statements of outputs 

3. The output map 

4. Knowledge, Skill or Sensitivity code for Enablers 

5. Code for Self-Other report 

With the above information in hand, a category number for each Work 
Requirement statement is to be determined by the coding team. This 
coding is directly related to each statement of output. Work Require- 
ment data are directly linked to specific outputs by way of the inter- 
view process, and the reliability of coding must therefore duplicate 
these condition^: by providing Work Requirem^'-nt statements in relation 
to specific outputs. As an end result of coding, category numbers will 
be determined for: 

A. Primary Standard Category number 

B. Primary Task Category number 

C. Cluster Task Category number 

D. Primary Enabler Category number 
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lirA-L-- Copsi st ency Data Analysi s 

In determining the consistency of the coders, individually or as n 
team, three basic comparisons can be made. Tlie first step in making any 
of the* three basic comparisons is the tabulation of the number of "cor- 
rect” items, i.e., th'^se codes which match the criterion code for each 
narrative statement , and the number of "wrong” i terns . \^en these quan- 
tities have been determined for each coder or team, the appropriate 
cons is tency comparison may be computed. 

One type of comparison involves the determination of deviation 
between the percentage of coder agreement with the criterion and an 
arbitrary percentage of agreement which indicates high consistency. This 
comparison can be handled in two ways. The most direct method is to re- 
quire each coder to exceed the arbitrary percentage of agreement. The 
other method is to compute a score which gives the standard deviation be- 
tween the rater’s percentage of agreement and the arbitrarily selected 
percentage of agreement. This 2 score can be interpreted by entering a 
normal curve table and determining the probability of a deviation that 
large occurring by chance. The Z score is used as a test of significance 
is used, but the significance levels considered acceptable are lowered 
from the usual .03 and .01 levels. If the Z value is too large, the rater 
is judged to be inconsistent with the criterion. Most elementary statis- 
tics texts include directions for computing the Z score and determining 
levels of significance. 

A second comparison, which provides a determination of test-re test 
consistency, utilizes the Chi-square statistic. The frequency tallies 
which have already teen prepared as described above for the pre- and 
post tests are combined in a 2 x 2 frequency table such as that shown in 
Figure 12. Once the Chi-square value is computed, its value, and the 
appropriate degrees of freedom, may be used to enter a Chi-square prob- 
ability table to determine if the distribution of correct and incorrect 
responses are sufficiently similar to say with some assurance that the 
two tests were coded consistently. A low Chi-square value with a high 
probability of chance occurrence indicates that the differences between 
the distribution of correct and incorrect responses for the two tests 
are very similar. One caution must be noted here. A low Chi-square may 
be obtained in two ways. Either the coder has consistently correct 
responses, or consistently incorrect responses. In both cases a low 
Chi-square value nay result. Therefore, the Chi-square value must be 
interpreted carefu-ily since a low Chi-square value when the responses 
are predominantly incorrect indicates a consistent coding performance. 

This test-retest situation tray be extended to more than 2 tests, in 
which case a K x 2 Chi 5quarj value must be computed, where K is the 
number of tests. 



correct incorrect 

Pretest 1 
Pcsttest 2 

Figure 12. Chi-square matrix for pre- and po.sttest results. 



K/ . 



62 



A rhird comparison is one wl.ich compares 2 or more raters on a 
single occasion. iliis coii^iarison can be made in reference to agree- 
ment v^ith the 'riterion or with r'ference to the agreement between 
ralerjJ on the coded statements. In either case a K x 2 Cai-square 
value is computed, where K is the number of raters to be compared 
vS i n«u Itaneously . The same interpretations and cautions apply to this 
comparison tnat were mentioned abovr for the second type of comparison. 



Activity Block 14: Computer Storage of Data in Prototype Data Bank 



Output Index ; 

P-52 Prototype Data Bank ( component , management) 

P-53 Data File Specifications (facilitating, management) 
E-54 Data File Creation (component, production) 

Output Map ; 
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P-52 Prototype Data Bank 

In any activity designed to generate the quantity of data implied 
by this methodology, the manipulative capabilities of a computer system 
are required. Design problems are at once apparent when it is observed 
that the extent and form of the data being collected remain unknown as 
long as categories remain open and subject to change. The possible com- 
b ina tions of data which migh t need to be availab le to answer newly con- 
ceived questions are also unknown. It is necessary, therefore, to 
build an ''unsophisticated** computer system capable of dealing with these 
uncertainties. Emphasis is placed on the design of data iiles which pro- 
vide maximal flexibility. In effect, the computer programs required, do 
little more than sort, a task which, in some cases, might best be per- 
formed by standard sort/merge types of routines. In short, the proto- 
type data bank requires the "design of a computer storage and retrieval 
system capable of storing and retrieving an unknown kind and amount of 
data to be used in as yet undetermined ways .** 

P-53 Data File Specifications , and 
P-54 Data File Creation 



Once the data have been coded they may be placed op data coding 
forms to facilitate the transfer of the data to an appropriate machine 
readable form (i.e., punch cards, p£per tape, or magnetic tape). De- 
tailed descriptions of the content of each data file are contained 
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within Appendix 8. All files created by the Oregon Studies were se- 
quential in form due to machine constraints, llie deficiencies in dara 
retrieval resulting from this approach could be corrected through the 

use of randomly accessible files which wo< 1 reduce retrieval tines 

considerably. Tne Appendix 8 listing of the files include the uam<- s of 
the variables, their order of presentation, their format, the type of 
data they repres 2 nt, and the work fuiiii and question numbers from which 
the variable values are derived. As a general rule the data should be 
checked digit by digit after it has been placed on ^he coding forms and 
before it has been converted :o the machine readabl'i fc.n. 



More specifically, the follcving procedures ^iiould be followed in 
converting the data to a machine readable form: 

1. Project descriptor data (Level 1) from Form 02 should be 

, placed on the data coding sheets utilizing the spacing 

described ■ ) Appendix 8. Tliere will be one record for 
each project and these records should be placed in ascending 
order by project number (Variable 1) Each of these records 
will consist of 63 variables and will be represented by 129 
digito. 

2. Personnel descriptor data (Level 2) from Form 03 should be 
placed on the data coding forms utilizing the spacing de- 
scribed in Appendix 8. This data set will consist of one 
record for each person interviewed and these records should 
be in ascending order, first by pioject number (Variable 1) 
and then by personnel number (Variable 2) within each proj- 
ect. Each of these records will consist of 135 variables and 
201 digits. 

3. Output descriptor data (Level 3) from Form 05-B should be 
placed on the coding sheets utilizing the spacing described 
in Appendix 8. There will be one record for each output 
identified within each of the projects. These records should 
be in ascending order, firs^: by project number (Variable 1' 
and then by output number (\ariable 2) within each project. 

Each of these records will consist of 17 variables and 35 
digits . 

4. The Scaled data (Level 4) from Form 04 should be placed on 

the coding forms as depicted in Appendix 8. Here, as at Level 
2, there will be one record for each person who is interviewed. 
These records will be ordered the same way as those at Level 2. 
Each Level 4 record will consist of 99 variables and 106 
digits . 



Standards descriptors (Level 5) from Form 06 are to be put on 
coding sheets in a form which differs slightly from the Appen- 
dix 8 descriptors. Because there may be multiple standards 
for each output, a record of this data set consists of the 
entries on a coded Form 06 page. This special format consists 
of descriptor data common to all of the coded standard state- 
ments for an output, along with up to 20 coded standard state- 
ments. If more than 20 standard statements appear with an 
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output > a ne;W record is begun with a reiteration of the conimon 
data. Usinpj this special format there is a reduction in the 
chances of miscopying a portion of ccmraon information, it is 
vital to inrure that uhis ^oinnion information is correct be- 
cause an eiror at this point ma} be multiplied up to 20 times 
in the final data set after the data conversion program has 
acted upon it. One data record contains all stand .rds data, 
up to 20 statements, for one output. This date set is sorted 
first by project number (Variable 1) and then by personnel 
number (Variable 2) within each project. 

6. Task descriptors (Level 6) from Form 07 are to be placed on 
the; coding sheets according to the Appendix 8 format. The 
general format for this data set is similar tc that used for 
thr Level 5 data in that a record consists of data common to 
all of the tasks on a form page along with up to 20 coded task 
statements from the form page (all task statements for one 
output/record, up to 20 task statements). These records are 
to be sorted first by project number (Variable 1) and then by 
personnel number (Variable 2) within the project. 

/. Enablers descriptors (Level 7) from Form 08 are to be placed 
on the coding sheets. The general format for this data set 
is similar to that used for the Level 5 and 6 data. These 
records are to be sorted first by project nuiiiber (Variable 1) 
and then by personnel number (Variable 2) with each project. 

8. Three "new" files must now be created in addition to the 

seven basic data sets. T'hese "new" files mereiy consist of 

copies of the Standard, Task, and Enabler data sets which 
are to be re-sorted. They must be sorted by project number 
(Variable 1) and then by output number (Variable 3) within 
a project, instead ^f by personnel number within a project. 

This process provides the 10 data sets needed fer complete opera- 
tion by the main program. Each of these 10 data sets must now be used 

as input to the data conversion program PRECON (Appendices 2 & 3) which 

takes each datci set and performs the necessary operations to produce 10 
binary mode data files which will be used by the main program. For the 
Project descriptors data set (Level 1) and the Personnel descriptors 
data set (Level 2) the program selects the variables marked by an 
asterisk in Appendix 8 and writes binary records which contain only 
these variables. This set of records, called Level 1-b and Level 2-b 
are the so-called "sort" sets. 

The Standards data sets, Task data sets and the Enablers data 
sets, Levels 5, 6, and 7, respectively, are expanded by :he program. 

The common information on each record of the original data sets is 
written in binary along with each of the coded Standard, Task, or Ena- 
bler statements. For each record in the original data sets, the data 
conversion program will produce up to 20 binary records, one for each 
coded statement. 

The 10 binary data files produced by this program provide the data 
input for the main program and its sorting routines. 
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Output Index : 

E“55 Verifying the Computer Inputs (component, management) 
E“56 Confirming the Logical Consistency of Items Coded to a 



E-55 Verifying the Computer Inputs 

Following the translation of data into card or tape format and the 
subsequent listings of the cards and/or tape, the listing is verified 
with the source documents. It must be kept in mind that for certain of 
the data elements a single input error can be magnified many times as a 
function of the amounts of other data elemints linked to it. For 
example, an output having 50 work requirement data elements linked to 
it and improperly entered as a research output would result in the 
listing of those 50 elements within the wrong categoiry. 

E~56 Confirming the Logical Consistency of Items Coded to a Category 

Once the computer inputs are verified, the first opportunity for 
examining the integrity of the classification of items to various cate* 
gories presents itself • Pro cedu rally, this simply involves the selec- 
tion of each category for which such verification is desired, and pre- 
paring a listing of all items coded to that category using the raw 
recapped statemen t s or labels contained in Forms 06, 07, 08, or 05 -A 
as appropriate. In some cases, a computer printout will facilitate this 
task by identifying the location of each of the relevant statements or 
labels . 

Once a listing is prepared, a visual ^content analysis** of the 
items can be made to assess the appropriateness of each to the category. 
There are three probable explanations for the inclus ion of items that 
appear inappropriate; (a) errors in computer input ; (b) errors in coding 
or recording the code; or (c) inadequate or inappropriately worded 
statements or labels that lead to misunderstanding of the item in the 
context of the category in which it is listed. 

Given the appearance of an item that ''looks out of place," the 
following procedures may be followed: 

1. Identify the location of the original documentation of 
the item (i.e., recap sheets. Forms 05-A or 05-B) . 

2. Compare the code given that item with the code of the 
category being examined. Discrepancy at this point 
indicates an error in computer input. 



Category ( component , management) 



Output Map : 



E-56 



E-55 





3 . If there is no disc ve pan cy at s tep 2 , re t'er the i tern 
for verification of coding. A necessary chan,?,e at this 
point indicates an error in original coding or record- 
ing, and a correction of the computer input is made. 

4. If the computer input and item coding are verified, 
the item must again be reviewed in the context cf its 
occurrence (i.e., project, outputs, etc.), and a judg- 
ment made as to the adequacy of the wording used to 
describe the item. Any changes made at this cime are 
solely for the purpose of more clearly descrioing the 
item. by providing such clarity, a potential reader 

is better able to understand why the item was classified 
as it was . 

Upon completion of the activities within this block, preparations 
for the processing and display of data are complete. 



Activity Block 16: Analyzing and Displaying Data Ac 7 *oss Cases 



Output Index ; 

P-57 Computer Program Descriptions (facilitating, management) 
E-58 Computer File Manipulation (component, production) 

P-59 Control Card Descriptions (facilitating, management) 

P-60 Examples of Retrieval Requests (component, production) 

Output Map ; 



P-60 



E-58 



P-57 



I P-59 



P-57 Computer Program Descriptions 

Once the data have been coded and entered into an appropriate 
storage medium, the user is ready to retrieve portions cf these data 
to help answer questions about educational RDD&E. 'Ihe types of ques- 
tions that may be asked are limited primarily by the ingenuity of the 
person asking the ques tions . To facilitate this ques t ion-asking 
process, a computer program was developed which allows a user to dis- 
play data which he feels are necessary to answer his question. This 
program consists of a main program (RDDE7) and several subroutines and 
functions which perform various specialized tasks. 

These subroutines and their respective tasks are as follows: 

1. Subroutine SORT performs a sorting function on records 
in a particular data file by matching user Hupplled 
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(TIUtIom a^alriHt Hclected variabk'K in the* data net and, 
i I a maLr.h uccur.H, wricing the record on a scratch ii It* . 

Subroutine I'AHl.KF takes the data from the scratch llie 
produced by SORT and, with instructions provided by tlu* 
user, prepares contingency tables of one, two, or three 
dimensions . 

3. Subroutine MKAN4 is a special purpose routine for one of 
the data sets which computes means, standard deviations, 
percentage of "zero’* responses, and percentage of responses 
greater than thre® for each variable. 

4. Subroutine HEADER reads an alphanumeric array which is used 
as line printer identification for ach set of tables 
reques ted. 

5. Functian ISO computes an appropriate subscript in a single 
dimension from multiple dimensions in order to more effi- 
ciently utilize available core sortage , since the GDC 3300 
does not support variable array sizes. 

6. Function LIMIT returns the maximum value of a variable for 
any variaDle in any data set. These values are used by 
function ISO to compute the subscripts . 

Appendix 6 contains listings of each of the computer programs 
developed for the Oregon Studies. Appendix 7 contains flow charts for 
each of these programs. 

E-58 Computer File Manipulation 

The main program RDDE7 uses seven basic data files which may be 
viewed as being, roughly, in a triangular arrangement as illustrated 
in Figure 13. The first level, or apex, of the triangle consists of 
project descriptors derived from Form 02 information. These variables 
and their sources are shown in Appendix 8. 



Level 1-b 



^ - - 

Le/el 4 



Level 2-b 













f 


1 Level 5 



Level 3 



Leve 1 6 | 



Leve 1 7 



Figure 13. Levels of data files. 
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From tills apex, the user has a choice of going to either of two 
files. These are labeled as Levels 2 and 3 in Figure J3, respectively. 

1. Level 2 is a set of personnel descriptors and, like Level 1, 
this level has two sets, lliere is a '’complete*' set of vari- 
ables which have been derived from information contained in 
Form 03 and there is a "SORT" set which is a shortened ver- 
sion of the "complete" set. Tliese sets are shown in Appen- 
dix 8 for the "complete" and "sort" (asterisk) versions. 

2. Level 3 is a set of output descriptors which are taken in 
total from Form 05-B and are shown in Appendix 0. From 
either of these two data files the user may choose to go to 
the other data set at the same level , go to any of the lower 
levels (4, 5, 6, 7) or go back to l^vel 1. 

The base of the triangle consists of Levels 4, 5, 6, and 7 which 
are derived from Forms 04, 06, 07, and 08, respectively. 

3. Level 4 data are shown in Appendix 8. These data consist 
of responses to Hemphill scale items which describe tasks 
that people in educational RDD&E may engage in as part of 
their jobs. 

4. Level 5 data pertain to standards inforruation coded on Form 
06 (Appendix 8) . 

5. Level 6 data pertain to task information coded on Form 07 
(Appendix 8) . 

6. Level 7 data pertain to enabler data coded on Form 08 
(Appendix 8) . 

P-59 Control Card Descriptions 

To illustrate the process of displaying information using this 
program, several examples will be followed through in detail. These 
examples will show the use of tne major options in the program, and 
also will illustrate che possible interconnections between the various 
date files. Before the examples, however, a general outline of the 
required and optional card image records and the variables represented 
on each is gi ven. 

The first record (A), which is required, is an alphanumeric array 
of up to 120 characte rs c tart ing in column 1 , This array is read by 
subroutine HEADER, is printed at the beginning of each table, and is 
used as a specific identification for a particular set of tables. The 
next record (B) , indicates to the program the level of data on which 
it will next be performing its functions. The variables represented on 
thin record and their tormat are as follows: 

!• Level to be sorted (12) . Variable one level to be sorted can 
take on values between 01 and 07. If values other than Lhesr 
are entered the program will stop execution or abort, dc^pruid- 
i ng on tho value . 



6 ? 






/. h url to bii performed (12) . VtiriabJu t.wu is an Indicator 

varlnbh*. if tte value is greater than zero, the user is in- 
dicatin^* that he has one or more var iable/levol combinations 
which he wishes to hold constant at this level. This type of 
sort will be referred to as ''real" to differentiate it from a 
"nominal" sort in which all variable/level combinations for a 
particular level are used. 

3. Logical unit number of tabling instructions (12) . Variable 
three indicates that tables will be made from data at this 
level if the value of the variable is greater than zero and 
the specific value given by the user is used by the program 
of the logical unit number for the file containing tabling 
ins tructions . 

Cross-sort between Levels 2 and 3 (12) . Variable four is used 
to indicate that the user wishes to perform sorts on both 
Levels 2 and 3. If this type of sort is desired, after a sort 
on Level 1 is called the next record indicating the sort level 
must have this variable set equal to one, to indicate the 
first part of the cross sort , and the next record indicating 
the sort level must have this variable set equal to two. If 
no cross sort is to be performed, this variable is to be set 
equal to zero. If this variable is set at zero, the next 
record indicating a sort level must either be one, or it must 
be four or greater. 

5 . Perform means and percentage analysis if this is Level 4 (12) . 
Variable five is set greater than zero when variable one is 
set at four to indicate that the user wishes to perform the 
means and percentages analysis on the Level 4 data (the only 
place this analysis is appropriate). 

The next record (C) is an optional one. Records cf this type are 
used only if variable two on record B is greater than one, indicating 
that the user wishes to hold some variable/level coubinations constant. 

1. Number of variable/level combinations to be controlled (13). 

2a. Variable number (13). 

2b. Level code (13). 

Repeat 2a and 2b until there are as many pairs as indicated in 
variable one. Varxable one appears as the first three digits and 
variables 2a and 2h are repeated with up to 20 pairs on each card image. 

At this time up to 50 variable/level combinations may be controlled 
at each level. When the user wishes to identify a new set of tables 
and start a new series of sorts, a record of type B, with variable one 
set equal to zero, will cause the program to function as if , were 
starting execution from scratch by rereading a new line of ht^ader 
information. To make the program function properly (if the user is not 
interested in "real" sorts on Level 1 and Levels 2 or 3), ''nominal" 
sorts MUST be made at Level i and either Level 2 or Level 3. This is 
necessary to provide the program with the information that it needs to 
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perform ’’real” or ’’nonunal” sorts at Levels 4 throuf;h / . At these 
lower levels, unless tables arc to be made, ’’nominal” sorts sliould not 
be made, since they serve n(> iisefuJ purpose. 

If variable three on record type B is greater than zero, indica- 
ting that some tables are to produced at that level of sort, then 
tabling instructions must be included on the logical unit indicated 
by the value of variable three. The information needed to produce 
tables is as follows: 

1. Var iable number of rows (13). 

2. Variable number of columns (13). 

3. Variable number of levels ( 13) . 

4. Option 1 (print 1-dlmens ional table as row table) (13). 

5. Option 2 (compute and print summation level) (13). 

The fir 5i t tiiree of the preceding variab les provide the program 
with the number of the variables which the user wishes vn occupy the 
positions of rows, columns, and levels . If the output being produced 
is one-dimensional, the user has th*e option of having eitner a row 
table produced across the page or a column table producer down the page. 
If variable four. Option 1, is greater than zero, a single-dimensional 
row table is produced, i.e., spaced across the page. If this variable 
is blank or zero, then the table is printed down the page and all cate- 
gories that have a frequency tabulation of zero are tict printed. In a 
similar manner, if a two-dimensional table is being produced, any row 
which has a frequency of occurrence of zero over all categories (row 
total is zero) is nc t printed, and in a three-dimensional table any 
level that has all zero entries in its cells is also deleted from print- 
ing. 



In order to make tables more readable it is sometimes necessary 
to print them somewhat differently than they may appear in a report. 
Since the printer page is relatively narrow compared to its length, it 
is good practice to use variables with large numbers of categories in 
the row or level positions rather than the column position. This is 
because the row doubles back on itself and multiple lines are printed 
for each row with more than 20 categories which makes reading the table 
more difficult. 

Any number of tables may be made at each level of sort. The tables 
produced are numb e red in consecutive order as they are produced within 
one request. Each time the user uses the 00 code on a B type record, 
the numbers placed on the tables are restarted at zero. Care must be 
taken when matching the tables to the list of tables requested, since 
any time a^ sort reveals that there are no records which meet the user- 
supplied criteria, all tables which were to be printed for that set of 
data are deleted and an appropriate error message to that effect is 
printed on the output. Also, if this condition of no records mee ting 
criteria was detected at either Level 1, 2, or 3, any sorts, and there- 
fore table sets, for Levels 4, j, 6, and 7 are deleted, and messages to 
that effect printed, until a record of type B is encountered which 
sorts at a level either lower than or at the sare level as that for 
which there were ”no records.” If the ”no record” condition occurs at. 
either Level 4, 5, 6, or 7, only that set of tables is deleted. 

■ . 

In k 



r-^)0 I . X .imp 1 i 'S of Kf'lrlrvnJ l^ oques tH 

In nrdpr in illuHtraU’ the ,>roc:c8fi of *1 i sp 1 ov i ii)* I n i nrinat i on , 
auverol (.‘Xat'ip Itns will be* shown in dutali. Since tin* :ouL portl(jn ol 
this pro^'ram may be* very time consuming at (;*ltlu*r levels 4, 5, (», or /, 
it Ls advantageous to carefubly plan the order and level of sort so 
til at a minimum numb^^’r of sorts are required. Tlio saim* [loJds true for 
tabling. Sometimes the use of a second or third dimension in a tabic* 
may eliminate the need for several sorts. 

The first example is one in wliich task information is desired 
about those having a Doctor's degree working In rcsoarcli-focused proj- 
ects. This request for information will require the nst* of informa- 
tion from three data files. Level 1-b indicates ihai variable two 
shows the focus of the project. Level 2-b indicates that variable 
seven shows the highest degree a person holds, and Level 6 contains 
task, information. Assume that the information of interest is a listing 
of the frequency of occurrence of a specific task category (variable 
12) which is desired for each major task category (variable 10). The 
header record needed to produce this information Ls as follows (these 
examples do not illustrate exact spacing, the reader is advised that 
this will depend upon implementation constraints): 

Logical Unit 45 
Col . 1 

EXAMPLE— 1 
0101000000 
001002001 
0201000000 
001007001 
06000J0000 

Logical Unit 01 

012010 (variable 12-rows, variable lO-columns) 

(file mark) 

This sequence of instructions will produce a single, two-dimensional 
table with variable 12, specific task category, as rows, and with vari- 
able 10, major task category, as columns. 

For the second example suppose that information was desired about 
master’s and bachelor’s degree level personnel working in development- 
focused projects who are working on outputs charac terized as research . 
The information desired is level of output and focus of output, each 
compared with major task categories, major enabler categories, and major 
standards categories. The header information for this request is as 
follows (again the reader is cautioned that the spacing is not exact 
and only for illustrative purposes) : 



Logical Unit 45 



(1) 


EXAMPLE-2 — 


~CROSS-SORT-BE‘n>lEEN-LEVELS 


-2-AND-3 


(2) 


0.101000000 


("real" sort, 


Level 1) 


(3) 


001001002 






(4) 


0201000100 


("real" sort , 


Level 2-cross to 3) 



TASK INFORMATION-FOR-PHDS-IN-RESEARCH-FOCUSED-PROJECTS . 

(calls "real " sort at Level i) 

(sort variab le 2 , category 1) 

(calls "real" sort at Level 2) 

(sort variable 7, catego’^y 1) 

(calls "nominal" sort and tables on LUN 01) 







(5) 002007003007004 

(6) 0301000200 

(7) 001016001 

(8) 0500290000 

(9) 0600290000 

(10) 0700290000 
(file mark.) 



("real" sort, Level 3-froir 2) 

("nominal" sort, tables on LUN29) 
("nominal" sort, cables on LON29) 
("nominal" sort, tables on LUN29) 



Logical Unit 29 

010006000000000 (variable 10-rows, variable 6-columns) 
010007000000000 (variable 10-rows, variable 7-columns) 
(file mark) 



This set-up will produce six tables, two each for standards, tasks, and 
enablers. The same tables may be produced in another way by changing 
records (4), (5), (6), and (7) to: 

(4) 0301000100 

(5) 001016001 

(6) 0201000200 

(7) 002007003007004 



The end result is the same, but this way sorts out research-focused 
products first, then sorts out bachelor's and master's level personnel. 
Also, records (8), (9), and (10) could have been placed in any order 
with the only change in output being the order in which the tables are 
printed. If any one of the sorts at either Level 1, I, or 3 had shown 
a condition of no records cieeting the criterion, then no tables would 
have been produced in this example. If, however, there had been addi- 
tional sorts at either Level 5, 6, or 7 and the condition of no records 
meeting criterion was encountered, then only the tables for the level 
showing no records would have been deleted. 



For the third example, suppose that information was needed to com- 
pare tlie responses on scaled data and task data for each of the foci 
that a project may take on, and then, in the same run, suopose that 
for evaluation-focused projects it was desirable to compare major task 
categories for each output function. The header records for this would 
be as follows: 



(1) EXAMPLE-THREE PART- 

(2) 0101000000 

(3) 001002001 

(4) 0200000000 

(5) 0400000001 

(6) 0600290000 

(7) 0101 

(8) 001002002 

(9) 02 

(10) 0400000001 

(11) 060029 

(12) 0101 

(13) 001002003 

(14) 02 

(15) 0400' 0001 

(16) 060029 



— SCALE D-AND-TASK-DATA 

(research projects) 

(means and percentage analysis 
(tables on LUN29) 

(development projects) 
(diffusion projects) 



on Level 



4) 
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(evaluation pr(^ ts ) 



(\/) )lf)l 
(\H) OOl'J(i.'004 
( 

(20) U4U00U0U01 

(21) 000029 

( 22 ) 00 

(23) KXAMPLE-TOHE 

(24) 0101 

(25) 001002004 

(26) 0301 

(27) 001017001 

(28) 060030 

(29) 0301 

(30) 001017002 

(31) 060030 

(32) 0301 

(33) 001017003 

(34) 060030 
(file mark) 



— PART-2 MAJOR-TASK-BY --OUTPUT-FUNCTION 

(evaluation project? ) 

(output function - policy) 

(tab les on LUN30) 

(output function - management) 
(output function - producclon) 



Logical Unit 29 
006 

007 

008 
010 

(file mark) 



(output level) 

(output focus) 

(output major category) 
(maj or task category) 



Logical Unit 30 

010 (major task category) 

(file mark) 



In the first part of the example it is necessary to return to Level 
1 for each new set of tables, since one of the variables of interest 
was project focus. A "nominal** sort is required at Level 2 each time. 
Records (5), (10), (15), and (20) show the form necessary to call the 
means and percentage analysis for Level 4. Record (22) illustrates the 
use of the 00 sort level code to introduce a new set of tables. Part 2 
of the example shows that it is possible to use one sort at a given 
level. Level 1 in this case, and then do repeated sorts at lower levels, 
holding the firs': sort constant for all further sorts. This example 
also illustrates »“hat there is an alternate path from Level 1 to either 
Level 4, 5, 6, or 7. 

These examples illustrate the uses of various options available in 
the program as well as the requirements of the program which, when 
followed and used properly, allow a wide range of data displays to pro- 
vide information about educational RDD&E . 

In addition to these computer-generated data displays, there is a 
possibility for other data displays derived from the data which were 
not computer stored and from other computer processing of already stored 
data. In the former case, this generally takes the form of hand tallies 
and/or summaries '^ata on the forms, which are not easily categorized 
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for romputer usage. Included In this category would be the summary of 
all of the *’other’* categories for the various questionr. In the latter 
case, this takes the form of possible statistical analyses. For in- 
stfince, if a two-dimensional table indicates differences in the distri- 
bution of frequencies, then it might be useful to perform a Chi-square 
analysis to sot* if the perceived differences are statistically signifi- 
cant . 



On a more sophisticated level, a Q-type factor analysis can be 
performed on the data stored in Level 4. By correlating all possible 
pa.^rs of people over their standardized responses to each variable, the 
correlations may be used as input to a standard ractor analysis pro- 
gram to derive factors, which represent hypothetical people and, hope- 
fully, may be identified in a reasonably concise manner. 

Other analyses may also be performed depending on the questions 
which need to be answered and according to the resources available. 



Activity Block 17: Computer Output of Profile Tables 



Output Index ; 

P-61 Computer Generated Profile Tables (component, production) 
Output Map ; 

f'P-6 1 I 

P-61 Computer Generated Profile Tables 

Approximately 50% of any profile presents and interprets the '*hard'* 
data obtained from that site (case). Though many different tables 
could be generated for each site, it is most practical to standardize 
the data displayed. In this way, comparison of the sane type of infor- 
mation from many sites can be made. 



There were six tables generatea for each of the 20 sites profiled 
for the Oregon Studies project. In future uses of this methodology, 
the use of these six standard tables is recommended as being feasible. 



1. Standards data — Since there are two major categories of 
standards statements, two two-dimensional tables are 
generated; 

(^) Output identification X Output standards (Set J-l/LM) 
(Variable 3 X Variable 12) 

(M Output identification X Processes /Operations standards 
(Set J-2/LM) (Variable 3 X Variable 12) 

2. Task data — Task categories are listed under 20 major head- 
ings (major task categories). One two-dimensional table is 
generated as follows : 

(^) Output identification X major task categories (Set NO) 
(Variable 3 X Variable 10) 




3. ICnablcrs data -- Enablers data consists ot information on 

knowledges, skills, and sensitivities used in the accomplish- 
ment of outputs. Three two-dimensional tables are generated 
as follows: 

(a) Output identification X Category set S-l/lJV 
(Variable 3 X Variable 12) 

(^) Output identification X Category set S-2/UV 
(Variable 3 X Variable 12) 

(O Output: identification X Category set S-3/UV 
(Variable 3 X Variable 12) 

The above si:; tables are printed out by the computer upon inter- 
rogation of the data bank for the variables as noted above. 

These printouts are delivered to the Coordinator of Data Inputs and 
Storage for subsequent delivery to the profile writer. 

If other data are of unique importance in profiling any particular 
site, special printouts can be requested by identifying the variables 
to be displayed and requesting or interrogating the data bank for the 
required tables. 



Activity Block 18: Coordination of Data Handling 



Output Index : 

C-62 Coordinated Handling of Data (component, management) 
Output Hap ; 



C-62 



C-62 Coordinated Handling of Data 

The volume of data to be collected and handled as part of the 
distribution and processing of data inputs is such that its flow must 
be coordinated throughout the data collecting agency. This coordinating 
effort is necessary to assure efficient data retrieval and routing for 
use among agency personnel. Maintenance of data location records and 
follow-up to assure completion of data inputs, storage, reductions, 
analyses, and reporting are the primary tasks of the Coordinator of 
Data Handling. 

The paragraphs which follow discuss the specific tasks involved in 
data handling. They are ordered sequentially and are expected to 
result in the output shown above. 

1. Retrieve from the site selection team the Project Contact 

Record (Form 01) and the Project Selection Form (Worksheet 01) , 
which should be attached to Form 01. If available, the Proj- 
ect information from Project Officer Form (Worksheet 02) 
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should also be obtained from the institutional representa- 
tive who made the initial site contact. 

2. As contacts are made with the site they should be recorded 
on the attachment to Korm 01 and filed with it. The contact 
should be recorded by the person maid ng it, and he should 
give the attachment to the Coordinator of Data handling for 
filing. 

3. Record the retrieval of Form 01 on the Chart of Project 
Information Status (Worksheet 18), assign an identification 
number to the project and record this number on the form as 
well as on the Project Identification Coding Chart (Worksheet 
04). Also on Worksheet 04, fill in the other information 
required about the selected site . 

4. Distribute a copy of Form 01 to the institutional representa- 
tive selected to visit the site. (Note: The institutional 

representative, the visiting team for a particular site, and 
the dates of visitation will be identified by the site-selec- 
tion team.) 

5. File the original Form 01 and any extra copies, including 
attachments as they are received. (See step 2 above . ) 

6. After the institutional representative has returned from his 
onsite visitation, retrieve from him the Project Questionnaire 
(Form 02), the Product Interview Form (Worksheet 09), the 
auxiliary materials according to the Dissemination Brochure 
(Worksheet 08), any contextual cue items (Worksheet 11), the 
project proposal, and any notes, charts, etc. derived by him 
while onsite. Confirm that item //3 on Form 02 is marked as 
either 1, research ; 2 , development; 3, diffusion; or 4 , 
evaluation . 

7. Record the retrieval of these completed data forms on Work- 
sheet 18 and apply the required codings. 

8. Duplicate seven copies of Form 02, Worksheets 09 and 11, any 
notes, charts, etc., and each auxiliary item collected accord- 
ing to Worksheet 08, including the project proposal. 

9. Distribute to the institutional representative and to each 
interview team member copies of all data input forms and 
materials collected by the ins titutional representative . 

10. File the original data input forms and materials, including 
any extra copies . 

11. Using the data inputs collected to date, an initial output 
map will be formulated under the direction of the team leader, 
the profile writer, and the Data Collection Coordinator. This 
initial output map should be retrieved from the team leader 

2 or 3 days after the data distribution described in step 9. 
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12. Record the retrievt-l of the initial output map on Worksheet 18 
and code it according to the project . 

13. Duplicate r.even copies of the initial output map. 

14. Distribute copies of the initial output map to the Data Manage- 
ment Coordinator, the institutional representative, and each 
interview team member. 

15. File the original and any extra copies of the initial output 
map . 

16. List any incomplete data inputs to date and provide the team 
leader with this information. This will allow for its collec- 
tion by tha team during site visitation. 

17. Immediately after the Interview team has returned from its on- 
site visication, retrieve from each member his uncoded inter- 
view recap forms (Forms 06, 07, 08), Job /Task Inventories (Form 
03), General Position Activities Questionnaire (Form 04), inter- 
view tapes, information on interrelationships, and contextual 
cue iteros. Also, retrieve the final output map from the team 
leader. 

18. Record on Worksheet 18 the retrieval of the data inputs from 
the interview team, and provide these inputs with the required 
code numbers. Also, on the Project Personnel Coding Chart 
(Worksheet 05) record identification numbers for each of the 
interviewees . 

19. Duplicate seven copies of each of the items retrieved from 
the interview team, excluding the interview tapes. 

20. Follow up to assure complete retrieval of all data inputs, 

since data retri'aval should Le complete at this time (see 
Worksheets 16, i/). List any incomplete items and provide 
the Data Management Coordinator and the team leader with this 
information. (Note: As incomplete data are retrieved down 

the line, duplicate, distribute, and store each item as 
described within these procedures.) 

21. Distribute to the profile writer the following data inputs as 
each listed itam is retrieved from the site. Copies of each 
should be assembled and distributed to him with a list of 
what is enclos 1. 

(^) Form 01, Worksheets 02, 03. 

(b) Form 02 . 

(£) All auxiliary materials collected according to Worksheet 08. 

(^) All Form 03 ^s. 

(£) All Form 04 *s. 

(_f) A copy of the final output map. 

(^) All contextual cue items collected according to Worksheet 11. 

(_h) All uicoded Forms 06, 07, 08. 

(_i) A]1 job descriptions. 
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(j^) All infonnation on project inter relationships . 

(jc) Any notes, diarts, etc. derived by data collection agency 
personnel wliile onsite, 

{ 1 ) Training Suggestion Form for each interviewee (Worksheet 13) , 

(m) Form 05 -A. 

22. Distribute to the coding resolution team the original Forms 

06, 07, Ob for purposes of coding the interview data. Distrib- 
ute to the data analysis section one copy of each of the items 
in the following list. When all the listed items are retrieved 
from the site, copies of each should be assembled for use in 
data analyses. Include a listing of the following items: 

(^) Form 02. 

(K All Fom 03’ s from the site. 

(jc) All Form 04 ’s from the site. 

(d) Form 05-B. 

23. File all the original forms retrieved from the interview team, 
any other data inputs from the team, and any extra copies of 
these items. 

24. Within one week after the interview team has returned from its 
site visitation, retrieve from the coding personnel the coded 
interview recap sheets (Forms 06, 07, 08). 

25. Duplicate the coded interview recap sheets (Forms 06, 07, Ob) 
and file these duplicates. 

26. Sort the original Forms 06, 07, 08 for the data analysis section 
in order to reference recap statements . These recaps should 

be bound as a single volume, whose identification number will 
be the same as that used on the project’s profile, and ordered 
as follows : 

(^) Sort all sheets into the three form number sets of 
06, 07, 08. 

(_b) Within each form number set, sort the pages in numerical 
order by interviewee identification number. 

(^) Starting with the numeral 1, consecutively number each 
sheet through to n for the 06 forms. Do the same for 
07 and 08 forms. Each form number set should then be 
numbered 1 through n. 

(^) Bind these sets into a single volume \;itU the 06 forms 
being placed first; then the 07 forms followed by 08 
forms . 

(^) The statements found on the recap forms should then be 
coded for computer storage in order that they may be 
located in the bound volumes of recap sheets by project 
number, form number, and page number. 

27. Distribute to the data analysis section the original, coded , 
and bound Forms 06, 07, 08 from the site. 

Retrieve from data analysis the reduced data. 
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28. 



Duplicnto five copies . 











29 . 


Duplicate five copies . 


30. 


Distribuue copies of reduced data to team Jeader and profile 
wx'iter . 


31. 


E'ile original reduced data and any extra copies. ‘ 


, 32. 

b 


Two weeks after the interview team returns from the site, 
retrieve the initial case-profile draft from the profile 
writer and duplicate eight copies. 


33. 


Attach appropriate cover letters (see Worksheets 20, 21, 22) 
and distribute copies of the initial case profile to each site 
interview team member, the site profile writer, the project 
coordinat:ors, the profile editor, and the institutional repre- 
sentative to the site for editing purposes as specj fied in 
each specific cover letter. 


. 34. 


File the original draft and any extfa copies. 


35. 

r 


After 2 days, retrieve profiles and comments from reviewers, 
compile them, and distribute to the profile writer. 


; 36. 


Three weeks after the interview team returns from the site, 
retrieve the final profile from the profile writer and 
distribute one copy to the profile editor for review and 
necessary revision. 


37. 


Retrieve and duplicate five copies of the editor's version of 
the final profile, file the original and any extra copies 
after mailing one copy to the site project director with 
directions for revision (see Worksheet 23). Provide for 
another coordinator's review, also. 


38. 


When retunxed, compile comments from site project director 
and project coordi nators . 


39. 


Distribute these to editor and writer, who together finalize 
the profile. 


40. 


Retrieve final version of profile from the writer. 


41. 


Prepare the profile for printing, have it proofed in final 
form, printed, distributed, and filed. 


O 

ERIC 

1 “ 
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Chapter 3 



CASE PROFILE PREPARATION 



This chapter of the methodology manual describes the writing, 
editing, reviewing, and distributing procedures for preparing a case 
profile of an educational research, development, diffusion, or evalu- 
ation project. Activity Block 19 identifies each of the major com- 
ponents of a profile as an output, describes the content of that com- 
ponent, and discusses how each of these components might be prepared. 
Because of the number of people Involved In the preparation of a pro- 
file, Activity Blocks 20 and 21 Include a discussion of the editorial 
and review procedures necessary for the control of the profile’s con- 
tent and quality. Finally, Activity Block 22 discusses procedures for 
printing and distributing a case profile. 

Because this chapter comprises a guide for profile writers, it 
has Its own set of attachments serving as examples of profile compon- 
ents. These attachments ave included within Appendix 9 of this 
manual. 



Activity Block 19: Writing a Case Profile 



Output Index * 

P-63 Introductory Pages of Profile (component, production) 

P-64 Profile Chapter I: Overview (component, production) 

P-65 Profile Chapter II: Parameters of the Project (component, 

production) 

P-66 Profile Chapter III; Summary of the Data (component, 
production) 

P-67 Profile Chapter IV: Supplementary Data (component, 

production) 

P-68 Profile Chapter V: Project Dynamics (component, production) 

P-69 Profile Chapter VI: Implications for Trairing (component, 

production) 

P-70 Profile Appendices (component , production) 

P-71 Initial Case Profile (component, production) 



Output Map : 




■^2 




P-63 Introductory Pages of Profile 



The introductory pages consist of a title page and the table of 
contents. The pu::poses of these pages are 1) to identify the specific 
profile so that it can be referred to as a discrete document, 2) to 
identify the project (and its parent agency) which is the subject of 
the profile, and 3) to provide an index to the profile. These pages 
are of a standardized format iind wording, but may be adapted as neces- 
sary to the specific profile. A sample of the title page appears as 
Attachment 1, and a sample of the table of contents is found as Attach- 
ment 2 of Appendix 9. 

The following shews content items and sources of information for 
each part of the title page: 

Item Sources 



1. 


Full title of observed project. 


1. 


proposal. Form 02 


2. 


Proj ect acronym. 


2. 


proposal. Form 01 


3. 


Agency name and address. 


3. 


proposal. Forms 01, 02 


4. 


Quote displaying main thrust 
or philosophy of project. 


4. 


interview tapes, 
proposal 


5. 


Legal statement about sponsorship 
of descriptive study . 


5. 


sample title page 


6 . 


Profile number. 


6. 


generally in sequence 
in which projects 
were described, but 
not necessarily 


7. 


Profile date. 


7. 


date of site visita- 
tion (month and year) 


8. 


Identification and address 
of agency describing the 
observed project. 


8. 


sample title page 


The quote displaying the main thrust or philosophy of the project 
(Item 4) can be selected from either the interview tapes or from the 
project proposal. Care should be used to select a quote which gives a 
brief, quick reference to what the observed project is about. A per- 



sonal rather than a formal statement should be sought, or any prevailing 
character or philosophy reflected, which helps a reader to focus on this 
project as a vital process having useful outputs rather than as a set of 
statistics or a section in a data bank. 

The table of contents should display information about the major 
sections of the profile, listing all subparts of interest to readers, 
and indicating all appropriate page numbers. 
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P-bA Profile Chapter I: Overview 



Chapter I is an overview of the project under consideration. Since 
many readers will probably read only the overview, rather than the pro- 
file’s entire detailed text, its intent is to s >rve as a complete ab- 
stract of the project. Generally not more than six pages, this chapeer 
should maximize the reader’s global perception about the project by 
providing a synopsis of the project, its objectives, rationale, signifi- 
cance, and the context in which it operates. 

Each of the parts of Chapter I is shown below with its associated 
data source. 



Item 



Sources 



1, Synopsis of the project 



2. Objectives, rationale, and 
significance of the project 



1, proposal, agency and project 
description from institu- 
tional representative, 
interview tapes. Form 02 

2. proposal, reports of iusLi- 
tutional representatives, 
interview tapes 



3. Context in which the project 
operates 



3. output mapj interview tapes, 
project time lines, con- 
textual map, project organ- 
izational structure chart, 
proposal 



This section is the reader’s first view of the observed project as 
more than a title. It must provide an understanding of the nature of 
the project, its goals, and its reasons for being, giving the reader a 
framework into which he can fit all the data contained iu the rest of 
the profile. The data in each profile are site specific. This section 
must make that site comprehensible and discrete. 



The ”S 5 mopsis of the Project” is a one-page summary and follows 
the format described in Attachment 3, Appendix 9. 



The ’’objectives. Rationale, and Significance of che Project” in- 
cludes no more than two or three pages covering: 



1. Primary goals and objectives, (main thrusts) 

2. Stated philosophy or rationale for the procedures, which may 
include a summary of the approach/procedures . 

3. Significance of the effort, particularly in light of the 
total program in which the project fits, or the priority 
needs of the sponsor/tiser . It should highlight the per- 
ceived value of the study. 
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In short, this section provides minimal elaboration of the thrust 
and reasoning behind the project, and concisely states the substantive 
information based on project documents and interview clarification. 

Also, it should indicate any special target group orientation for the 
project (e.g., special education, K-6 reading, etc.), and the nature 
of likely consumer groups for the outcomes of the project. 

The ‘‘Context in I'lhich the Project Operates” summarizes in about 
two or three pager the following; 

1# Relationship to other agencies. 

Around the contextual map (see Attachment 4, Appendix 9), 
it identifies the major aspects of the project’s: 

(£) Relationship to parent agency (and any intermediate units, 
including relevant supportive resources /specialists of 
the agency, such as a computer facility). 

(^) Relationship to funding sponsor(s), 

(_c) Relationship to immediate user agency (s) (with whom 
coordination is required) , 

(d) Relationship to subcontractor (s) (including relevant 

supportive resources /specialists of the subcontracting 
agency) . 

(£) Relationship to other ongoing project(s). 

(_f) Relationship to consortium members. 

(^) Relationship to advisory group(s) . 

(h^) Relationship to potential user(s) (groups or persons/ 
agencies to be informed of project efforts). 

2. Relationship to other efforts of an overall program. 

Identified or briefly discussed here are the project’s 
relationship to preceding, concurrent, and/or anticipated 
succeeding phases of an overall larger program effort, if any, 
and: 



(^) Any uncertainty regarding the continuation or termina- 
tion of this project . 

(^) Any uncertainty regarding the initiation of anticipated 
follov-on projects, implementation, or other efforts. 

3. Supporting and technological resources. 



This identifies the more significant technological 
resources that are relevant to the project, if not covered 
in its "Relationship to other agencies," Included would be: 



(_a) 

(y 

(c) 

(d) 

(e) 



Support (or 
Support (or 
Support (or 
Support ^r 
Support (or 



resources) 

resources) 

resources) 

resources) 

resources) 



provided 

provided 

provided 

provided 

provided 



agency . 

by user agency (s) . 
by subcontractor(s) . 
by other project(s) . 
by consortium members. 






4. 



Time lines . 



Around a ^project time lines** figure, the significant 
functional activities shown are discussed. Briefly identified 
are any major constraints, advantages, pressures, etc. Also 
briefly indicated is the rate of progress (or delays) up to 
che time of interview and those anticipated in the future. 

Major obstacles or factors that impede /enhance progress are 
cited, such as redirection of efforts as the s tudy progresses , 
lack of staff or resources, morale crises, availability of 
prior materials for immediate use, etc, 

5, Physical/environmental setting. 

The more significant characteristics of the work setting 
including the following (as appropriate) are identified here: 

(^) Necessity to carry out the work in more than one location, 
(y Geographic location, 

(£) Adequacy of work setting with respect to disturbing 
influences • 

Subsections of this chapter with their related source materials are 
shown below: 



Item 

1, Time lines (figure) 

2, Relationship to agency and 
intermediate units 



3. Supporting and technological 
resources 

4. Contextual map (agency 
structure chart) 



5. Project and/or agency 
management pattern 

6. Interrelationships with other 
agencies, groups , etc , 

7. Physical /environmental 
setting 



Sources 

1. project time line chart, 
proposal. Worksheet 11 

2. representative reports, 
interview tapes, agency 
documents. Forms 01, 02, 03, 
Worksheet 11 

3. Form 03, proposal, inter- 
view tapes 

4 . representative reports , 
interview tapes, agency 
documents. Forms 01, 02, 
Worksheet li 

5. Forms 02 and 03, Worksheet 
11 

6. Form 03, interview tapes, 
proposal and other project 
documents 

7. Worksheet 11 



•' lij Itil’yi'f H, p rMpt^'T.j J , y (jur UI'IIMI I. h , 

|/i’w^raiii t'llorts ('ph/isc^s, etc.) inlervIiiW Laptib, reprL'.scjiu- 

ative reports , Forms 01., 02 

P-65 Profile Cha pt er IL: Parameters of the Project 

Chapter II einrhasizes the “hard” data gathered around the project ’r> 
structure and the outputs it is generating. It includes the interrela- 
tionships found betr^een outputs and between staff members. It should 
make these data quickly and easily available to the reader. Neither 
conclusions nor interpretive discussions about the effects of these 
data should be provided. Such information will be provided later in 
Chapters IV and V. Chapter II indicates what is innovative about each 
output, as applicable. It also describes how the project and/or ou“:put 
designs are unique or unusual, and presents the project staff’s view 
of the focus of the project and its outcomes. Included here will be: 

1. Project structure. 

(^) Staff structure . llie staffing patterns, with infor- 
mation about selection and structure, should be presented in 
both a narrative and a tabular form. Around the “Project 
organizational structure chart” (see Attachment 5, Appendix 9) 
are des^’.ribed the staff’s various functional positions and roles. 
This should include the significant positions and roles of 
others , such as subcontract personnel , representatives or 
monitors from user agencies , advisors / consultants , and other 
participants. Described briefly are the operating interrela- 
tionships between these project groups and their functional 
roles- The supportive resources/specialis Ls available within 
the project are also described . 

“Project personnel” is defined here as including all 
persons working to meet the funded goals and objectives, in- 
cluding subcontractor personnel, advisors, and other partici- 
pants; but excluding higher agency officials, except as .they 
are directly and actively involved in the project - The figure 
may include current full time equivalency for each position. 
Briefly discussed are any staffing features that seriously 
affect the project, such as deviation from desired staffing 
and personnel turnover. 

Indicated also are how the staff structure may or has 
changed rormally over time, such as the departure of concep- 
tualizers and designers after their aspect of the job is done, 
or the need for production specialists in the future phases of 
the project. 

(^) Project roster . The primary information provided by 
the project roster is concerned with the type of work roles 
which exist on the project. Supplementary information will 
vary from profile to profile, reflecting for each particular 
staff some aspect which the writer feels is important for under- 
standing the functioning of that particular project. Tied into 
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the '‘Project roster of staff by job titles" is the staff struc- 
ture description. A sampl'^ is Included as Attachment 6, and 
Incorporates full time equivalency figures. 

The sample shown displays a unique featuve of the subject 
site which the writer felt Influenced how, how well, anc} how 
fast the tasks of the project were done. In this case, the 
majority of the staff had worked together before on a similar 
project. On other sites, the formal training or previous work 
experience may be an unusual :;eature of that particular staff. 

For example, inexperienced persons may display different 
behaviors in attempting to accomplish a set of goals than 
experienced persons would display in working toward those same 
goals . 

This section would indicate which personrc?! were inter- 
viewed and identify the outputs with which each was associated 
in the interviews. The roster may be placed in the appendix of 
the profile if it gets too lengthy or disruptive in the chapter. 

2. Outputs Generated 

(^a) Index of outputs . A descriptive list of outputs is 
to be generated by the interview team. A sample of such a list 
is included here as Attachment 7. An introductory discussion 
should precede the list in a form felt by the writer as helpful 
toward the reader’s understanding of the list as it is presented. 
Generally, the presentation will include a brief description 
of each output, and an arbitrary identification number consist- 
ing of two parts: a letter which permits easy identification 
of the output as a product (P), condition (C), or event (E), 
and a sequence numeral irrespective of P, C, or E. Finally, 
those outputs interviewed around should be identified in some 
manner, e.g., by an asterisk. 

Included under this heading, also, will be a table which 
indexes the outputs by categories. Although outputs may be 
categorized in many ways, those generally used will be 
C^) orientation, i.e., product, event, or condition, focus, 

i.e., research, development, diffusion, or evaluation, 

(^) level, i.e., focal, component, or facilitating, and 

(^) stage of completion. An example of such a table is found 

as Attachment 8. 

(^) Output map . A final output map will be completed by 
the interview team and included in the profile as a figure. 

An introductory discussion should precede the map describing 
the interrelationships and dependencies existing between the 
outputs displayed. An example of such a map is found as 
Attachment 9 in Appendix 9. 




t ; 

t 
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iihsrc I 1 onH nf lIiIm oha[)ter» wltli ihu 1 r u'laLi'd ^^uvi^cA• malvrlahi, 
dmwn bo low: 



I com 

1. Final output map (a figure) 

2. De:icriptivo annotated listing 
of outputs identified 

3. Output ::ategories 3. 



Sources 

1. Interview team leader 

2. Form 05, supplementary papers 
Crom site such as interim 
reports and relationship 
notes, interview tapes 

Form 05, interview tapes, 
supplementary papers from 
site 



4. Staffing pattern 

(_a) staff selection and 
training 

(y organizational structure 
(a figure) 



4. 

(a) interview tapes , report 
from institutional rep- 
resentative, Form 03 

(_b) interview tapes, inter- 
view notes. Worksheet 11 



(jc) project roster of staff 
by Job titles 



(O institutional repre- 
sentative reports, job 
descriptions, tapes, 
proposal 



P-66 Profile Chapter III: Summary of the Data 



Chapter III is a summary of specific data generated about project 
outputs in terms of output standards , the tasks pertaining to them, and 
the knowledges, skills, and sensitivities that enable their attainment. 
It will include tabular presentations along with discussion about data 
organization and interrelationships . 



It should describe to the reader what the nature of the data is, 
how they were collected and organized, and how they are being presented. 
Included in this presentation are several tables showing the frequency 
of standards, tasks , and enablers pertaining to each output interviewed 
around. Described also is the system of coding sets as related to 
standards, tasks, and enablers; and how such categories are used to 
summarize the project data. The reader is referred to the listing of 
output standards, tasks, and enablers in the profile appendix for 
specific data statements. The sample size of informants on each output 
is indicated, as well as the trus tworthiness of the ob tained information . 
Information trustworthiness is a function of the informant *s Involvement 
with the output, his level of perspective as a function of his position 
within the project structure, and the length of recall or anticipation 
time regarding output activities. 



A table of output standards cited per interviewed output is included 
which provides the frequency with which specific standards occurred with- 
in the production standards categories. The standards categories are 






related to particular outputs by means of this table and subsequent 
interpre tive discussion . Some specific ill us t rat ions of s tandards coded 
within a category may be cited to aid the reader in understanding what 
a category represents . 

Other relationships should be noted as appropriate, such as the 
relations between standards and the structures of an output. Other 
available data might: relate the influence of project stage of completion, 
of time lines, or of interview background to the occurrence of standards 
citations by interviewees . The significance of selected standards should 
be pointed out and described. Important relationships of some standards 
to specific tasks might be stated. A display of the proportion of 
standards that overlap between categories of outputs would also be use- 
ful in some instances. 

Included also is a process /ope rations s tandards table that provides 
the frequency with which specific standards occurred within the process/ 
operations s tandards categories . These s tandards cate go? ies are related 
to particular outputs by means of the tab le and subsequent interpre tive 
discussion. Some specific illustrations of standards coded within a 
category may be cited to aid the reader in understanding what a category 
represents. Other relationships should be noted, as deed in the section 
above for output standards. 

The preceding tables and procedures can be used for displaying 
tasks, knowledges, skills, and sensitivities. These tables are not the 
only possible display of the data available. A variety of other stand- 
ard data displays may be obtained from the computer data bank. 

P-67 Profile Chapter IV; Supplementary Data 

Chapter IV contains the remainder of the **hard" data on the project, 
consisting mainly of the summaries of questionnaire items. Some of the 
possible items to go into this chapter are listed below. If the data 
become too unwieldy, they can be placed in an appendix. 

1. Output differences (by type) across the life of the project. 

2. Summary of staff background. 

This would present the data from Form 03, the Job/Task 
Inventory. The table could be broken up into portions of 
inventory if not used in previous chapters, such as: 

(^) Personnel background (i terns 1-13) 

(M Prior work experience (items 14-23) 

A portion of Form 02 also provides some staff data (items 
14-15). 

3. Summary of interviewee responses on Form 03. 

(^) Present position requirements (items 24-30) 

(M Support resources (items 31-38) 

(^) Project management (items 39-42) 
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4. Summary of interviewee responses on Form 04, General Position 
Activities Questionnaire , 

F-68 t^roiile Chapter V; Project Dynamics 

Chapter V is a discussion of any special conditions affecting the 
project, the work being done, or the quality of its outputs. It prin- 
cipally is a chance for the writer to react somewhat subjectively to 
the project in terms of the interrelationships influencing its opera- 
tions and results. The dynamics chapter provides an opportunity for 
the writer to ”let loose" and tell the reader what it is like to live 
inside of the project. There should'^U^ n'O restriction in the writing 
of this chapter nor any particular structure in it. The reader should 
be provided with as much information as possible about the influences 
of: (This list should not be interpreted as exhausting the possibili- 

ties for discussion within this chapter.) 



1. 


Staffing patterns 


6. 


Issues 


2. 


Management structure 


7. 


Staff backgrounds 


3. 


Management styles 


8. 


Agency interrelation- 


4. 


Commitments 




ships 


5. 


Affect 







Specific information on interrelationships can be gleaned from 
Worksheet 11 (Contextual Cue Items) and from Worksheet 14 (Interview 
Notes on Interrelationships) in addition to the interview tapes, 

P-69 Profile Chapter VI: Implications for Training 

Chapter VI sui\^.iar izes what the project information relates about 
training needs for professionals in educational research, development, 
diffusion, and evaluation. It describes the way in which job defini- 
tions (or functional roles) vary across time and/or interact with each 
other. Specific information on training can be gleaned from Worksheet 
13 (Training Suggestions Recap) and from items 22-30 on Form 03, In- 
terviewee responses might be interpreted in light of knowledge of their 
background (Form 03, items 4-23) , Comparisons are made between 
(^) Form 03 items 22-23 with items 24-25, and (^) enablers from Form 08 
with items 24, 25, and 30 on Form 03, If there are any training needs 
that are unique to personnel working with special target groups, these 
should be noted as v^ell, 

P-70 Profile App e ndices 

A set of appendices are attached to each profile to provide 
supplementary data for various readers. The purpose of this section 
is to provide the reader with additional information which would have 
been out of place in the profile due to length, interest only to 
special audiences , etc. 



Some possible appendices to the profiles appear below. This list 
should be altered, at the writer’s discretion, to fit the needs of the 
profile • 

A Project Roster of Staff by Job Titles (or In Chapter II) 

B Outputs of the Project (annotated or in Chapter LI) 

C Bibliography of Available Outputs (if ary) 

D Sample(s) of Output(s) Produced by Project (may be anno- 
tated if used) 

E Summary of Staff Background (or in Chapter V) 

V Summary of Interviewee Questionnaire Responses (or in 
Chapter V) 

G Summary of General Position Activities 

H Project Proposal (edited, if used) 

I Listing of Output Standards, Tasks, and Enablers 

J Historical Papers of the Project 

Although the profile as a whole is the responsibility of the 
writer, many of its components need to be generated by other agency 
personnel. The people responsible for various profile Items are listed 
in Attachment 10, Appendix 9. 

P-71 Initial Case Profile 



The result of the preceding set of activities described for Outputs 
P-63 to P-70 is an initial case profile. All of the parts and pieces 
of a profile have been collected, organized, and presented by the writer, 
but the document still needs to be worked and reworked until all con- 
cerned are satisfied that it truly represents the project described in 
an accurate scientific and literary fashion. The activities involved 
in honing the initial document into a final form are subfiequently de- 
scribed in Activity Blocks 20-22. 



Activity Block 20: Editing a Case Profile 



Output Index : 

P-72 Edited Initial Case Profile (component, production) 
Output Map : 



P-72 



Extensive editing of each case profile is necessary for quality 
control. The editing procedure described in this section interlocks 
with Activity Block 21 on profile reviews, since review of all aspects 
of the profile by persons involved with the project site and the data 
collection agency is as necessary to the editing process as is tech- 
nical editing in providing for comprehensive quality control. This 
section on editing a case profile specifies procedures .id personnel 
involved in controlling for the technical accuracy of the profile. Its 
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i. 



2 . 



3. 



4. 



5 . 



6 . 



7. 



8 . 



s to provide a description of the technical editing procedure, 
escribe areas of responsibility for the personnel involved in 
ess. 

ited Initial Case Profile 



following steps are involved in the editing process: 

When the writer has completed an initial ■:a?='e profile, it 
is given to che Coordinator of Data Handling for duplication 
and dis t r ibution t :> the pvofile editor. 

'line firsc review by the profile editor deals with the major 
technical problems of the profile draft. Since profiles 
are writi.en by different writers, and each writer generally 
follows his own style of writing, the profile editor is 
responsible for displaying what has been written in a proper 
profile format. Additionally he is responsible for spelling, 
grammar, etc., as well as general compatibility between profile 
writers. The instructions for the editor can be found in 
Attachment 13, Appendix 9. 

When the editor has completed this major, technical review 
of the profile, he returns it to the Coordinator of Data 
Handling who has it retyped, duplicated, and distributed 
within the data collection agency for review. (This review 
process 3s described in Activity Block 21: Reviewing a Case 

Profile . ) 

When each person reviewing the profile in step 3 has com- 
pleted his task, he returns the profile with his comments to 
the Coordinator of Data Handling. Upon receipt of each of 
these reviews, by a specified date, the Coordinator of Data 
Handling distributes them to the profile editor. 

The profile editor then assists the writer in collating 
these review coinraents and writing a final version of the 
profile. The final version is also returned to the Coordina- 
tor of Data Handling. 

The final version is duplicated by the Coordinator of Data 
Handling and sent to the site project director for review . 

(This process is discussed in Activity Block 21.) 

Once the comments by the project director regarding this 
final version of the profile have been returned, they are 
given to the profile editor, who, with the help of the writer, 
will make any necessary revisions. 

The profile editor then returns this final copy of the profile 
to the Coordinator of Data Handling for preparation as a 
suitable copy for printing and distribution. (This process 
is discussed in Activity Block 22.) 
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Activity Block 21: Reviewing a Case Profile 



Output Index ; 

P-73 Reviewed Initial Case Profile (component, production) 
Output Map : 



P-73 



As part of the process of editing each case profile th^re are two 
major reviews which provide for quality control in content, complete- 
ness, accuracy, and representativeness. The first is by staff within 
the data collection agency, and the second is by the director and 
staff of the project being profiled. ' 



P-7 3 Reviewed Ini tial Case Profile 



The following steps occur within the review process: 

1. When the editor has completed his first major, technical review 
of the profile, he returns the profile to the Coordinator of 
Data Handling who has it retyped, duplicated, and distributed 
within the data collection agency for review (see step 3 in 
Activity Block 20) . 

This inagency review is the responsibility oi (^) all agency 
personnel who visited the project site for data collection 
purposes^ i.e., data collection team members and agency rep- 
resentatives, and (^) the coordinators of the data collection 
agency. (It frequently happens that these positions overlap, 
e.g., a coordinator may also be a representative, or a repre- 
sentative may also oe a team member.) 

(^) Review by data collection team . This review allows the 
profile to be checked for accuracy of content ^ interpre- 
tations, and completeness by those persons who were di- 
rectly involved in the data gathering. First-hand exper- 
ience at the described site will allow this group to check 
or correct the profile for congruence with their own per- 
ception of conditions and interactions of the profiled 
project and its site. The profile transmission sheet, 
including specific areas of review responsibility, is 
Attachment 11 in Appendix 9. This sheet Is included 
with each copy o2 the review draft of the profile, and 
is to be used as a checklist of responsibilities by those 
persons to whom it is addressed. 

(b) Review by data collection agency coordinators . This 
review provides for an overview of the profile by the 
management personnel of the data collection agency. It 
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includes such areas as communicability to general audi- 
ences, and adequacy of technical language for more spe- 
cial! zed audiences . The ins tructions for the management 
review are Attachment 12 in Appendix 9. 

2. After the activities in steps 4, 5, and 6 occur in Activity 
Block 20, the director of the profiled project and his staff 
review the profile. Final approval of the content and inter- 
pretations within the profile, and approval for any use of 
project documents, e.g., edited versions of the project propo- 
sal, sample outputs, aiid historical papers are sought as a 
result of this review. 

A cover letter (see Attachment 14, Appendix 9) vdll accom- 
pany the profile sent to the project director for his review. 
This cover letter outlines the director’s review responsibili- 
ties and refers to two other documents . One is a form for 
rating the degree to which the profile represents the project, 
and the other is a profile release form permitting use of the 
profile as part of the output of the data-gathering effort. 
These t\;o forms are found as Attachments 15 and 16, Appendix 9. 

3. Upon completion of the profile review process, the next step 
is the same as step 7 in Activity Block 20. 



Activity Block 22; Printing and Distributing a Case Profile 



Output Index ; 

P-74 Final Case Profile (component production) 
Output Map ; 



P-74 



P-74 Final Case Profile 

At this point in the preparation of a case profile, the Coordina- 
tor of Data Handling should have a final copy of the profile which has 
been completely rev^iewed and edited (see step 8 in Activity Block 20). 

This copy of the profile must now be prepared for printing and distribu- 
tion. This may involve simply duplicating a few copies of the profile 
within the agency for limited distribution, or it may involve a sizable 
contract with a publisher for major printing and distribution. The 
many details that must be attended to in this process will vary depend- 
ing upon the decisions that need to be made. The one major responsibility 
of the Coordinator of Data Handling at this point is to be sure that the 
profile is in the copy form needed for whatever decisions are made about 
printing. 
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Conclusion 



Output Index : 

C-75 The Ability to Replicate Data with the Oregon Studies 
Methodology (focal, production) 



Output Map : 



C-75 



C-75 The Ability to Replicate Data with the Oregon Studies Methodology 

Output C-75 does not specifically belong to Chapter 3, nor does it 
represent any portion of any activity block in any of the chapters in 
this volume. However, all three chapters are designed so that replica- 
tion of the outputs and processes described will result in the user 
being able to generate additional data compatible with the data gener- 
ated by the Oregon Studies. This is as indicated in the output map to 
the volume, the focal output of the volume. 
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EPILOGUE 



The methodology described in the first three chapters of this 
volume was conceptualized and built to provide descriptive information 
about educational research, development, diffusion, and evaluation 
projects. A number of variables were devised and a means of collecting 
data about the variables described and operationalized. A vast amount 
of data was collected and procedures for its automated storage and 
retrieval designed and implemented. The methodology continued to 
evolve during the course of the project and was in fact being modified 
as this volume was being written. 

The reader might ask, after having read the first three chapters 
of this methodology manual, what utility such a procedure might have 
within the context of his own project or organization. The response 
to that question is as complex as was the development of this method- 
ology. The procedures as described in the preceding three chapters, 
when implemented according to the specifications presented in those 
chapters, provides for the user a means of describing a directed set of 
activities in terms of the major variables, i.e., outputs, tasks, ena- 
blers anci standards. This descriptive application of tlu: methodological 
procedures provides no value judgments about the data collected, al- 
though the summarized data will surely be assessed by the user once 
they have been collected. 

By focusing upon the output, the methodology is essentially asking 
the question ’’what is it that you wish to accomplish?’* It does not 
provide the means for achieving the output, by way of defining the 
exact nature of the process necessary for successful production of the 
output, nor does it speciry or delineate the variety of technologies 
already in existence within the professional educator’s ’’bag of tricks”, 
e.g., PERT, PPBS, behavioral objectives, etc. Instead the methodology 
provides the user with a systematic means of defining the exact nature 
of the actual outputs, tasks, enablers, and standards. Once such an 
exact description of ”what is” is produced, the user can determine if 
there is need for the invocation of specific problem-solving strategies. 

Based upon some value judgments which were made after visiting 
20 exemplary sites, a tentative hypothesis can be advanced: If an 

organization, group, etc. is to succeed in the production of some out- 
put in a most cost effective and efficient manner, with the highest 
degree of professional involvement and behavior, then it is critical 
for all staff members to know, and share in operational terms, the com- 
mon direction or goal; understanding not only his own contribution to 
the realization of that goal but the contribution of the staff members 
who compose the organization, project, etc. Even though no attempt was 
made to evaluate the individual sites which were visited, it was clearly 
the case that there were differences not only in the types of outputs 
produced (this is discussed thoroughly in Volume I), but in the manner 
in which the personnel on those sites approached the tasks at hand.' 
After all, it is people who create outputs. While the outputs may be 
well done, there was and is a difference of attitude in people’s 
approach to the task of producing the outputs they arc charged witli. 
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Obviously, describing a problem will not provide commitment or 
di rod ion on tlic part of a staff. The definition of tlie activities I z*om 
the viewpoint of all staff involved and the perspective gained through 
the reduction of the forthcoming data, however, provides a means lor a 
staff to examine themselves and decide for themselves if there is a prob- 
lem in existence, and quite possibly the appropriate steps to take to 
alleviate the problem circumstance. This is In contrast to systems 
analysis methodologies which frequently produce descriptions of problem 
states using any number of variables, but from the point of view of a 
third party. While this has been believed to be a desirable procedure, 
it would appear from our experience that a staff engaged in a set of 
focused activities gain much more if they are the ones doing the de- 
scribing, receiving a helping hand in asking the question of themselves, 
summarizing their thoughts, and discussing problems which may have been 
discovered. Once this has been done as a staff, discussing alternative 
problem solution strategies can take on the power and the support of a 
total group. This in contrast to the ^'legislating of PPBS," as an 
example. 

This is not to say that descriptions such as those presented in 
Volume IV necessarily lead to action. The implementation of alternative 
strategies which will alleviate any problem circumstances which may be 
discovered is still up to the users of these procedures . 

In short, the methodology as a descriptive tool can provide an 
organization, group of individuals, etc. with a thorough and complete 
description of what is actually being done in relation to the execution 
of a directed set of activities through the identification of the ac- 
tions and outputs, etc. Utilizing this methodology also provides the 
opportunity to visualize a set of activities from the point of view of 
the individuals actually engaged in them. Conversely it provides the 
individuals engaged in the work activities with a description of them- 
selves. These procedures summarize and organize the data provided 
by the individuals producing the outputs. This is of significance not 
only from the standpoint of accuracy of description, but also from the 
point of view that it helps clarify the contribution of every staff mem- 
ber to the directed set of activities, not only abstractly, but for each 
individual (a precondition to commitment?) . This methodology can provide 
a project or agency director a means of having individuals come to 
realize what their contribution to an overall goal or objective actually 
is. It can provide the vehicle for the identification of problem states. 
These descriptive attributes of the methodology" allow the user to paint 
a picture, frozen in time or dynamic through time, and on the basis of 
that picture diagnose immediate or potential trouble spots. 

The methodology, then, can provide an accurate description. A 
description of people doing something, engaging in behaviors which may 
or may not be directed toward a desirable outcome. 

In short, the Oregon Studies have provided the opportunity to think 
about in the conceptual sense, expand in the physical sense, and utilize 
in a time-dependent sense a methodology within a number of different and 
diverse settings. This version of the methodology would appear to be one 
of the most significant outputs of these Oregon Studies. 
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GLOSSARY 



ADOPTION 1. the taking of and using as one*s own. 2. in present usage, 
refers to the acceptance for use of KNOWLEDGE, INFORMATION, 
and/or TECHNOLOGY. 

CLUSTER 1. a number of different things grouped together on the basis 
of more general quali ties shared among them. 2 . in present 
usage, refers to a basis for grouping and classifying large 
numbers of specific OUTPUTS and TASKS, such groupings referred 
to as cl us ter categories. 

CHARACTER 1. the distinctive kind or sort that a thing is. 2 . in 
present usage, refers to a classification given an OUTPUT 
that is distinguishable as an instance of KNOWLEDGE, TECHNOLOGY 
IMPLEMENTATION, or INFORMATION. 

COMI^ONENT 1. a part or integral ingredient in a whole. 2. in present 
usage, a classification given an OUTPUT having such relation- 
ship to a FOCAL OUTPUT. See FOCAL and FACILITATING. 

CONDITION 1. manner, circumstance, or state of being. 2. in present 
usage, a STRUCTURE classification given an OUTPUT which 
represents an instance of a desired circumstance expected 
to endure within the life of a PROJECT, or beyond the PROJECT 
as a result of it . 



CONTEXT 1. the situation, background, and environment within which 

a specified set of focused activities occurs. 2. in present 
usage, '^focused activities” refers to PROJECT, and "situation., 
environment” to its FOCUS, SETTING, SIZE, CONTENT. 



DEVELOPMENT 1. a working out in detail or expansion of (a technique). 

2. in present usage, refers to: (a) engaging in a problem- 

solving strategy designed to produce reliable TECHNOLOGY, 
that is , procedures , materials, hardware , and organizational 
frameworks that have a known degree of success in bringing 
about a particular outcome or in performing a defined OPERA- 
TION; (b) a FOCUS classification given a PROJECT. 

DIFFUSION 1. the conveying of something from one person, place, or 
situation to another. 2. in present usage, refers to: (a) 

a problem-solving strategy designed to bring about the IMPLE- 
MENTATION of generalizable KNOWLEDGE, a reliable TECHNOLOGY, 
or trustworthy INFORMATION (as used here diffusion incorpor- 
ates the concepts of DISSEMINATION, ADOPTION^' and UTILIZATION); 
(b) a FOCUS classification given a PROJECT. 



DISSEMINATION 1. scattering far and wide. 2. in present usage, refers 
to the distribution of KNOWLEDGE, INFORMATION, and/or TECHNOL- 
OGY to a targeted population . 



l.Jji;(.Al J (Jh'AI. a coordinated set of problem-solving strategics 

designed to produce OUTPUTS that can be judged as to their 
quality and their contribution to the solution of educational 
problems . 

ENABLER 1. that which makes able, provides with means, power or 

authority. 2. in present usage, it refers to those KNOWL- 
EDGES, SKILLS, and SENSITIVITIES needed to produce a particu- 
lar oirrpuT. 

EVALUATION 1. finding the values or amounts of (a phenomenon). 2. in 
present usage, refers to: (a) engaging in a problem-solving 

strategy aesigned to produce trustworthy INFORMATION regarding 
a phenomenon which occurs in a context or environment over 
which the user expects to. exercise influence or about which 
he expects to make decisions; (b) a F0CU3 classification given 
a PROJECT. 

EVENT 1. happening or occurrence. 2. a STRUCTURE classification 
given an OUTPUT which represents an instance of occurrence 
of an observable transaction or set of behaviors. 

FACILITATING 1. making easier or assisting the performance or accom- 
plishment of. 2. in present usage, a classification given an 
OUTPUT having such relationship to and which is not an instance 
of a COMPONENT or FOCAL OUTPUT. See COMPONENT and FOCAL. 

FOCAL 1. of or pertaining to the center of activity or attention as 

a matter of contractual (implied or otherwise) obligation. 

2. in present usage, a classification given an OUTPUT having 
such relationship to the outcomes of the efforts of a PROJECT . 
See COMPONENT and FACILITATING. 

FOCUS 1. concentration: as, a focus on a kind of strategy. 2, in 

present »’sage, refers to a classification given a PROJECT hav- 
ing as the primary concentration of problem-solving work ef- 
fort a strategy characteristic of either RESEARCH, DEVELOPMENT, 
DIFFUSION, or EVALUATION. 

FUNCTION 1. a special duty or service rendered by a person or thing in 
the course of work or activity. 2. in present usage, refers 
only to a classification given an OUTPUT in accordance with 
the POLICY (setting), MANAGEMENT, or PRODUCTION service it ren- 
ders to the course of PROJECT work. 

IMPLEMENTATION 1. the carrying into effect and using of. 2. in present 
usage: \^a) a CHARACTER classification given an OUTPUT which 

represents an instance of the ADOPTION and UTILIZATION of 
KNOWLEDGE, TECHNOLOGY, and/or INFORMATION; hence, (b) an OUTPUT 
of DIFETJSION. 

INDEX 1. a list describing the items of a collection. 2. presently 
used in relation to an OUTPUT index , a listing of the OUTPUTS 
of a PROJECT. 
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INFORMATION 1. trustworthy facts or data descriptive of something with- 
out reference to interpretive or value judgments relating to 
that description . 2 • in present usage : (a) a CHARACTER 

classification given an OUTPUT which represents an instance of 
such facts or data; hence, (b) an OUTPUT of EVALUATION, 

KNOWLEDGE 1. empirically verifiable ideas inferred from facts or data 

which add to that which is known • 2 • in present usage : (a) a 

STRUCTURE classification given an ENABLER which represents an 
instance of that which is to be known as a requisite to accom- 
plishment of an OUTPUT; (b) a CHARACTER classification given 
an OUTPUT which represents an instance of such newly inferred 
and generalizable ideas; hence, (c) an OUTPUT of RESEARCH, 

LEVEL 1, a position within a dependent structure, 2, in present 

usage, level refers to one of three positions in the dependent 
ordering of OUTPUTS, i,e,, FOCAL, COMPONENT, FACILITATING. 

MANAGEMENT 1, the act or art of handling, orchestrating, and directing, 
2, in present usagvi, a FUNCTION classification given an OUTPUT 
which orchestrates the resources (time , pers onnel, materials , 
space, INFORMATION) available to a PROJECT in the realization 
of outcomes expected from it, or is a report of that orches- 
tration , 

MAP 1. a drawing or other representation of the relationships of, 

2, in present usage, (a) a schematic ordering of the OUTPUTS 
of a PROJECT in accordance with their interdependent relation- 
ships (OUTPUl map ) ; (b) a schematic ordering of the organiza- 
tional structures which exist within the CONTEXT of a PROJECT 
(Contextual map ) . 

OPERATIONS 1, any movement or series of movements made in fulfilling a 
purpose, 2, in present usage, refers to movements within a 
PROJECT in terms of the TASKS performed in the generation of 
the OUTPUTS for which the PROJECT is responsible, 

OUTPUT 1, something manufactured, created, or produced, 2, in present 
usage, refers to: (a) an identifiable outcome of targeted 

work activity which contributes to the realization of PROJECT 
goals; (b) a STRUCTURE classification given STANDARDS which 
serve as criteria for judging the adequacy of an outcome of 
targeted work activity, 

POLICY 1. any governing principle or guideline for a course of action, 
2. in present usage, a FUNCTION classification given an OUTPUT 
which establishes the principles or guidelines to be attended 
by a PROJECT, such classification frequently referred to as 
policy setting, 

PRIMARY 1, fundamental, elemental, 2, in present usage, refers to an 
initial categorization of interview data in which the category 
retains the fundamental and essential elements of each specific 
datum; hence, ^.timary categories of OUTPUTS and TASKS, 
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I’ROOiSS 


1. a continuing procedure generally composed of many steps 
and/or involving many changes. 2. in present usage, a term 
combined with the term OPERATIONS as a STRUCTURE classifica- 
tion given STANDARDS which serve as criteria for judging the 
adequacy of procedures associated with specified OUTPUTS. 


PRODUCT 


1. something tangible or "har« produced as an outcome of work 
effort. 2. in present usage, a STRUCTURE classification given 
an OUTPUT whici. represents an instance of a tangible outcome, 
concrete in form, and transportable at a given point in time. 



PRODUCTION 1. the act of fabricating or creating. 2. in present usage, 





a FUNCTION classification given those OUTPUTS which build 
directly on the fabrication objectives of a PROJECT. 


PROJECT 


a formally recognized , funded and directed effort aimed at 
achieving one or more specified ends that have their defini- 
tion in educational RESEARCH, DEVELOPllEN* ‘ , DIFFUSION, and 
EVALUATION. 


RESEARCH 


1. studious inquiry, examination, investigation, or experi- 
mentation aimed at the generation of KNOWLEDGE. 2. in present 
usage, refers to: (a) engaging in a problem-solving strategy 

designed to produce ideas, principles, theories, and laws 
that can be sho\;n to be generalizab le through empirical veri- 
fication; (b) a FOCUS classification given a PROJECT. 



SENSITIVITY 1. a quality of having perceptiveness and responsiveness. 





2. in present usage, a STRUCTURE classification given an 
ENABLER which represents an instance of a specific percep- 
tiveness and responsiveness considered requisite to accom- 
plishment of an OUTPUT. 


SETTING 


1. environment, background, or surroundings of a place. 

2. in present usage, setting refers to a classification given 
a PROJECT residing essentially in a college or university, 

R&D agency, private firm, or public school environment. 


SIZE 


1. extent, scope, or magnitude of a thing. 2. in present 
usage, refers to the current fiscal year funding and number 
of professional staff of a PROJECT, taken independently and 
in combination. 


SKILL 


1. ability, proficiency, or expertness in accomplishing some- 
thing. 2. in present usage, a STRUCTURE classification given 
an ENABLER which represents an instance of a specific ability 
considered requisite to accomplishment of an OUTPUT. 


STANDARD 


1. a grade of excellence or attainment regarded to be a meas- 
ure of the adequacy of something. 2. in present usage, a 
specific criterion applied to, excellence expected of, or a 
criterion by which judgments of adequacy are made about OUT- 
PUTS and PROCESS/OPERATIONS. 
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STRUCTURE 1. an orderly arrangement of the parts of a whole. 2. in 
present usage, refers to the constructs which identify the 
parts of each of several whol.iS, as in structure of OUTPUTS 
(PRODUCTS, EVENTS, CONDITIONS), structure of STANDARDS (OUT- 
PUT, PROCESS /OPERATIONS), structure of ENABLERS (KNOWLEDGES, 
SKILLS, SENSITIVITIES). 

TASK 1. any discrete piece or unit of work. 2. in present usage, 

a discrete unit of work performed among other similarly 
targeted units in producing or achieving a specific OUTPUT 
to the STANDARDS set for it. 

TECHNOLOGY 1. a technical method of or framework for achieving a 

practic.al purpose. 2. in present usage: (a) a CHARACTER 

classification given an OUTPUT which represents an instance 
of a reliable strategy, procedure, hardware, or set of 
materials designed to bring about a particular outcome or to 
perform a defined OPERATION; hence, (b) an OUTPUT of 
DEVELOPMENT. 

UTILIZATION 1. the making use of. 2. in present usage, refers to 

the making use of KNOWLEDGE, TECHNOLOGY, and/or INFORMATION. 
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APPENDIX 1 



Forms and Worksheets 



Forms 



FOl 

F02 

F03 

F04 

F05 (A+B) 
F06 
F07 
F08 



Worksheets 



Project Contact Record 
Project Questionnaire 
Job/Task Inventory 

General Position Activities Questionnaire 

Output Recap 

Standards Recap 

Task Recap 

KSS Recap 



WOl 
W02 
WO 3 
W04 
W05 
W06 
W07 
W08 
W09 
Wll 
W12 
W13 
W14 
W16 
W17 
W18 
W20 
W21 
W22 
W23 



Project Selection Form 

Project Information from Project Officer 
Project Selection Data 
Project Identification Coding 
Project Personnel Coding Chart 
Data-collection Personnel Coding Chart 
Travel Information 
Dissemination Brochure 
Output Interview 
Contextual Cue Items 
Interview Record Form 
Training Suggestions Recap 
Interviewer Notes on Interrelationships 
Checklist of Inter/iewee Data Inputs 
Checklist of Project Data Inputs 
Chart of Project Information Status 
Profile Transmittal Sheet (Team) 

Profile Transmittal Sheet (Coordinators) 
Profile Transmittal Sheet (Editor) 

Profile Transmittal Sheet (Project Director) 



APPENDIX 1 



Form 01 



Forms and Worksheets 



PROJECT CONTACT RECORD 



PRIME CONTRACTOR 



Project it 



1 . 

2 . 



5. 



Project Acronym: _ 
Project Director: 
Position Title: 
Address: 

Telephone: 



Other Project Contact: 
Contracting Agency: 



Agency Director and Tltle:_ 
Address : 



Telephone: 



Sponsoring/ Funding Agency: 

Agency Department: 

Address ; 



Project Officer; 

Position: 

Telephone: 



Other contact and/or protocol linkages, partlculary for purposes of 
providing any required clearances: 
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APPENDIX 1 (contM) 



FOl-2 



SUBCONTRACTOR Project it 

(include others involved in project sucn as demmstration 
sites, u^er -ngenc^'es, v^oordinating projects, etc.) 



1, Project Acronyra:_ 

2. Project Director: 
Position Title: _ 
Address ; 



Telephone: 

3. Subcontracting Agency: 

Agency Director and Title: 
Address : 



Telephone: 

4, Other contacts and/or protocol linkages, particularly for purpose of 
providing any required clearances: 






APPENDIX 1 (cont'd) 



FOl-3 



COMMUNICATIONS LOG 



Project // □XI 



Project Acronym or other identifier: 



Date of Contact: 



Contact Was Initiated By: 



Person Contacted Was: 
Purpose of Contact: 



Comments : 



NOTE: This form should be completed each time a contact is made with 

a project, contractor, sponsor, subcontractor, or other project- 
related person, forward it immediately to the coordinator of New 
Data Inputs for filing. 



APPENDIX 1 (cont'd) 



Form U2 



Project It 



PROJECT QUESTIONNAIRE 



This questionnaire was developed by Teaching 
Research, a division of the Oregon State 
Systems of Higher Education, pursuant to a 
grant from the U. S. Office of Education. 



APPENDIX 1 (cont’d) 



F02 

l^roject , ^ [ I 



PROJECT QUESTIONNAlKi: 



INSTRUCTIONS; Check the Box j y^| opposite each 
item that applies to the project, or write in 
the data required. 



2. Official Project Title 



3, Primary Project Objective 



Director’s !.ame 




4. Month and Year of 
Data Collection 


5. Starting Date of 
Project 


h. Ending Date of Project 









7. 



Total Duration of the Project 


8. Type of Contracting Organization in Which the 


lJ 1 - 7 months 


Project Resides 


[j 8-11 months 


□ Education Research and Development Center 


D 12 months 


D Regional Educational Laboratory 


□ 13-15 months 


D College, School of Education, Psychology Dept., 


□ 16-23 months 


or other university office or agency 


□ 24-29 months 


Q Private Non-Profit Organization 


D 30-36 months 


Q Industry or Commercial Organization 


□ 37 months or more 


Q Public School or School District 




Q State Department of Education 




Q Professional Association 




Q Agency of the Federal Government 




(specify department) 




D Other (specify) 



9. Project funding from all sources 



a. Total b. Current c. Last Year 

Fiscal Year 



□ 

D 

□ 

□ 

□ 

□ 



□ 

□ 

□ 

□ 

D 

□ 



10. Sources of Total Project Funds 



No funding 

D $20,000 or less 

□ $20,001 - $100,000 

□ $100,001 - $225,000 

□ $225,001 - $500,000 

D More than $500 ,000 

Approximate Percent of 
Funds from Each Source 



a. 


D us Office of Education 


i. 




b. 


D Other federal agency (specify) 


j . 




c. 


D State (specify) 


k. 




d. 


Q Private foundation 


1, 




e. 


D Local school district 


m. 




f. 


CJ College or university 


n. 




g- 


O Industry 


o. 




h. 


Q Other (specify) 


p. 
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APPENDIX 1 (cont’d) 



F02-2 



1.1. Primary Source of Funds 



12. Number of Funding Sources 



13. Percent of total project expenditure for each fiscal year of project 

a. Current year % b. Last year % c. Next year 



14. What is the total number of personnel who regularly receive all or part of 


their 


salary (or other financial stipend) from the project: 












Current 




Anticipated 




Currently 


Fiscal Year Last Year 


Next 


Year 


Professional Staff (including 


a. 


f. 


k. 


P- 




management) 












Clerical support staff 


b. 


g- 


■I 


q- 




Technical support staff 


c. 


h. 


m. 


r , 




Sub-contractor professionals 


d. 


i. 


n . 


s . 




Other significant personnel 


e. 


j. 


o. 


t. 




(consultants , advisors , etc. ) 












15. List the numbers of various academic degree 


levels of 


your total current project 


staff. (Do not include consultants and sub 


-contractors . ) 








Professional Staff 


Support Staff 






Doctorate 


a. 




h. 






Specialist or Professional 


b. 




i. 






Masters 


c. 




:i • 






Bachelors 


d. 




k. 






Associate 


e. 




1. 






Technical License 


f. 




m. 






Other (specify) 


g. 




n. 








(this column 


(this column 








should equal the 


should equal 


the 






number i 


cited in 


number cited 


in 






item 14 


.a above) 


14 b & c above) 





16. Enter complete bibliographic information on each published report or substantive 
document of the project. (Continue on reverse side as necessary.) 
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Form 03 
Project // 
Personnel // 



JOB/TASK INVENTORY 



Instructions 



The purpose of this inventory is to gather information about the 
actual backgrounds and opinions of professional persons who perform 
educational Research, Development, Diffusion and Evaluation activities. 
This inventory has the sanction of your organization and project 
di rectors . 

Please respond as candidly as possible. Your responses will be 
kept confidential and the compilations made from these inventories 
will be reported only in statistical form. 

There are no right or wrong responses and no preferred pattern 
of responses. 

Mark your responses and comments clearly and leglb ly and make any 
supplementary notations regarding pertinent information in the margins 
or the writing spaces provided for "other” responses. 

Thank you. 



This questionnaire was developed by 
Teaching Research, a division of the Oregon 
State System of Higher Education, pursuant 
to a grant from the U.S. Office of Education. 
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APPENDIX 1 (cont'd) 



JOB/TASK INVENTORY 



ro3 

Project It 
Person !> 







□ 












1 



PERSONNEL BACKGROUND SECTION 






INSTRUCTIONS; Check the Box 
opposite each item that applies to 
you, or write In the data required. 



2. Short title or acronym for project 



1. Last Name 



First 



^ame — Middle Initial 



3. Your official title on this project 



A. Your functional job title, most descriptive of your work 
role on this project 



6. Age 

Cl Under 20 

□ 20-24 

□ 25-29 

□ 30-34 

□ 35-39 

□ 40-44 

□ 45-49 

□ 50-54 
D Over 54 



Highest Degree you hold 

U Doctorate or Post-D 
Q Specialist or Professional 

□ Masters 
CJ Bachelors 
Q Associate 

Q Technical License 

□ None of these 

□ (Spe cify other: 

) 



10. Major area of specialty for highest degree 



a. 


□ 


Education/Te aching 




s tudy 


b. 


□ 


Education Administration 


a. 


□ 


Education /Teaching 


c . 


□ 


Educational Research 


b. 


□ 


Education Administration 


d. 


□ 


Guidance/Counseling 


c . 


□ 


Educational Research 


e . 


□ 


Vocational Training 


d. 


□ 


Guidance /Counseling 


f. 


□ 


Statistics /Measurement 


e . 


□ 


Vocational Training 


g- 


□ 


Psychology 


f. 


□ 


S tat is tics /Measurement 


h. 


□ 


Engineering 


g- 


□ 


Psychology 


i. 


□ 


Computer Sciences 


h. 


□ 


Engineering 


j- 


□ 


English /Writing 


i . 


□ 


Computer Sciences 


k. 


□ 


Business Adminis tration 


j • 


□ 


English/Writing 


1. 


□ 


Other: 


k. 


□ 


Business Administration 








1. 


□ 


Other: 



5. Sex 



I 1 Male Qj Female 



8. Professional certification or licensing 

Type: 

Issuing Agency: 



9. Number of professional publications 
authored 

□ None □l-3 □ 4-8 

I I 9-15 □ 16+ 



11. Minor areas of speciality for college 



12. Your annual salary 

D Under $5,000 

□ $5,000-$8,999 

□ $9,000-$ll,999 

□ $12,000-$16,999 

□ $17,000-$21,999 

□ $22,000-$29,000 

□ Over $29,000 



13, National Professional Memberships 

a« D AERA (American Educational Research Association) 

b. □ NEA (National Education Association) 

c. D AVA (American Vocational Association) 

d. D APGA (American Personnel and Guidance Association) 

e. D APA (American Psychological Association) 

f. Q AECT (Association of Educational & Communications 

Technology) 

g. □ Other: 
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PRIOR WORK EXPERIENCE 



O 

ERIC 



Items 14 to 23 refer to your work experience prior to your present assignment on 
this project. 



14. 



a. 

b. 

c. 
cJ. 

e. 

f. 

g. 



Number of years of experience in each of the following: 

Teaching in a college or university 

Conducting research in a college or university 

Working in public schools 

Working for state or national educational agencies 

Working in educational R & D centers 

Working in present organization (may overlap with above) 

Other educational or research employment: 



15. Describe briefly what you did in each of the applicable employment areas noted in 

item 14. Indicate all of your work experience of the last 10 years, or after age 20, 



a-g 

Area 



Years 

Worked 



Descriptive 
Job Title 



Type of Work 



16. Total number of years of experience 
in educational research , development , 
evaluation , and/or diffusion (not 
including teaching) : 



18. On how many educational research, 
development, diffusion, or evaluation 
projects have you been Principal 
Investigator? 



20. How many educational research, devel- 
opment, diffusion, and evaluation projects 
have you worked on? 



17. Total number of years of managerial 
experience in directing educational research, 
development, evaluation, and/or diffusion 
projects? 



19. How many proposals for projects have you 
written or helped prepare, and submitted for 
funding consideration? 



21. How many funded projects are you now 
working on? 



22. What, in your professional training ^ was most relevant to providing you with the 
knowledges and skills necessary for the performance of your duties on this project? 
(That is, what specific courses, research, thesis, apprenticeship, etc.) 



23. What, in your past professional w ork experience > was most relevant for your 
preparation on this project? 



U 
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I'rom the licncfit of your experience and your perception of vour work wl tliin this project, 
pic.iso res])oncl to tlie following items from an idealized point of view. Consider the 
rtvjij i romen ts of your position a.s if you were hiring someone to fill that position. 



24. What specialized train'ng should be required? What particular areas of training? 



25. What specific knowledges and skills does the position require? (Describe as fully 
as possible.) 


26. How many years of voik experience does 
the position require in educational 
research, development, diffusion, and/or 
evaluation? 


27, How many years of work experience does 
the position require in administration 
activities? 


28. How many years of work experience does 
the position require in management 
activities ? 


29. What level of academic training does 
this require? 

D Doctorate 

Cl Specialist or Professional 

•-J Masters 
Q Bachelor 
dl Associate 
O Technical License 
Cl None of these 

(Specify if other: ) 


30. What in-service training have you 
undergone on this project to prepare you 
specifically for your current assignment? 
(For example, informal briefing, workshops, 
specific fomial courses, etc.) 







O 

ERIC 
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SI:PP0RT RiiSOURCES 



El. Please clieck all support services from the list below 


, provided by other persons 


or agencies , 


that you must have to carry out your job on this 


project . 


a . 


□ 


Equipment construction (mechanical, electronic, 


carpentry , etc . ) 


b. 


□ 


Print Lng 




c . 


□ 


Other reproduction services: 




d. 


□ 


Photography 




e . 


□ 


Art work and illustrations 




f . 


□ 


Drafting 




g* 


□ 


Technical writing 




h. 


ri 


Editing 




i. 


P 


Secretarial service, other than typing 




j • 


□ 


Typing 




k. 


u 


Purchase of supplies and equipment 




1. 


□ 


Library holdings 




m. 


□ 


Subscriptions to technical and professional journals /period! cals 


n . 


□ 


Requests for documents or publications not locally available 


o . 


Cl 


Computer analysis se rvices (Data processing) 




P- 


□ 


Computer program writing 




q- 


□ 


Statistical consultation 




r . 


□ 


Audio-visual aids and devices 




s . 


□ 


Subjects for experimentation or try-out of procedures 


t . 


□ 


Travel arrangements 




u . 


□ 


Budgetary and other fiscal accounting 




V . 


□ 


Scoring of test items 




w . 




Television facilities and equipment 




X . 


□ 


Other : 




y • 


□ 






2 . 


□ 














32. Please check all the support equipment listed below that are immediatelv 


available 


and 


used by you in this project. 




a . D 


Dictating equipment 




b. □ 


De: 


sk calculators 




H 


Desk- top computer 






Remote computer terminal 




n 

e. Cl 


On- 


-site computer 




f. □ 


Key-punch machine 




PI 


Data card sorter 




h. □ 


Photographic equipment 




1. □ 


Video tape 




.i- ° 


Television camera 




k. □ 


Readers for microfiche or microfilm 




1. □ 


Other significant equipment: 




m. D 








n. D 


















, 



O 

ERIC 
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J'3, WhaL amount of travel is 
expected of you on this project? 



34. To what extent are project manpower resource s 
(nuniber and capabilities) adequate? 



a. Number of trips per year 

□ Nc ne 

□ 

□ 3-5 

n 6-10 

□ n-15 

□ 16-25 

D More than 25 



b. Average duration of each tilp 
D No trips involved 
D Less than 1 full day 
n 1 day each trip 
D 2 days 
D 3 days 
O 4-5 days 
^ 6-8 days 

D 9-15 days 
n More than 15 days 



D Reasonably adequate 
D A bit tight 

□ Extremely short 



35. To what extent are project time lines and 
available period of pe r f o rmance^ aJeq ua t e7 

D Reasonab ly adequate 
D A bit tight 
D Extremely short 



36. To what extent are project financial resources 
adequate? 

Q Reasonably adequate 
□ A bit tight 
D Extremely short 



37. Is some reasonable amount of project time made available to project personnel for 
each of the following? 



Yes No 



a. 


□ 


□ 


b. 


□ 


□ 


c. 


□ 


□ 


d. 


□ 


□ 


e. 


□ 


□ 


f . 


□ 


n 


g- 


□ 


□ 


h. 


□ 


n 



38. 


Does 


your 




Yes 


No 


a. 


'n 


□ 


b. 


□ 


□ 


c. 


□ 


□ 


d. 


□ 


□ 


e. 


□ 


□ 


f . 


□ 


□ 


g- 


n 


□ 


h. 


□ ' 


□ 



Preparation of professional papers for publication 

Presentation of speeches at conventions or other professional meetings 
Attendance at conventions or other professional meetings 
Personal reading of current professional literature 
Reading of professional literature relevant to this project 
Preparation of proposals for possible funding of future projects 
Establishment and maintenance of contacts with other agencies, potential 
sponsors , etc. 

Interaction with personnel on other projects within your own organization 



project or agency expect you to engage in the following? 



Preparation of professional papers for publication 

Presentation of speeches at conventions or other professional meetings 
Attendance at conventions or other professional meetings 
Personal reading of current professional literature 
Reading of professional literature relevant to this project 
Preparation of proposals for possible funding of future projects 
Establishment and maintenance of contacts with other agencies, potential 
sponsors, etc. 

Interaction with personnel on other projects within your own organization 



/. p * 
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PROJECT MANACEMENT 



y) . Check all of the following which are usually provided to you by the project leader 
and/or his key subordinates? (Project directors should answer these items from the 
viewpoint of how they provide project guidance.) 



Yes Mo 



a. □ □ 

h . □ □ 

C. G G 

d. u 

e. □ G 

f. □ □ 

g. □ □ 

h. □ □ 

i. □ □ 

J . □ 



Detailed project time lines. 

Specific forecasting of project efforts, and associated monitoring and 
accounting (such as by a PERT system). 

Specified work assignments on a daily-to-weekly basis. 

fieneral statements of functional goals, to serve as a basis for individual 
work assignments or responsibilities . 

'"'ccasionai reviews of effort, and general discussion of remaining work 
goals . 

Frequent review of work achievement. 

Weekly or bi-weekly staff meetings to review project progress. 

Annual or semi-annual review of individual performance and effectiveness. 

Close daily working contact. 

Other means by which you know what you should be doing on the project 
and how we"l it is being done: 



AO. Project structure as viewed by project personnel: 

n Pyramid (based on span of control and delegation of responsibilities, without 
much staff support from parent agency) 

n Corporate structure ( wi th many supporting staffs and resoiirces of the parent agency) 
D Inverted pyramid (few workers, but many advisors, consultants, and/or evaluv.tors) 

D Chain of command (vertical linearity, with many levels of command) 

D Leaderless (cooperative work team, with general reporting to parent agency) 

Q Other: 



41. To what extent is coordination required 
within this project? 

D 1-Very minimal coordination 
□ 2 - 
L 3- 

Cj 4-Moderate amounts of coordination 

□ 5- 

□ 6 - 

D 7-Very extensive and involved 
coordination 



42. To what extent does this project require 
coordination with other agencies (schools, 
sponsor , sub-contractors , consortium 
members, related projects, etc.)? 

Q 1-Very minimal coordination 
□ 2 - 
□ 3- 

D 4-Moderate amounts of coordination 
H 5- 
n 

L-* 7-Very extensive and involved 
coordination 
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' ’.'iryilQl WOR K ASSIC.:'^1EI ":S ^ 

A'!. If you are now working on other funded projects, in addition to this project, 
i wlicil functionai job titles are most descriptive of your work role on cm’li of tlic 
(itlier projects? Indicate all current Job roles you perform within the agcuiry, and j 
the position titles. 

t 

i 

I 

I 



44. How many persons do you supervise directly? 

a. Currently, on all projects : c. Normally, on all projects: 

b. Currently, on only this project: d. Normally, on this project: 

45. How many persons work under your general supervision, though not necessarily 
under your direct supervision? 

a. Currently, on all projects: 

b. Currently, on only this project. 



46. l^at is your relationship to the primary contractual project? 

I j Regularly a part of the prime project staff. 

An on-call member of the prime agency, performing on the project as needed. 

Regularly a part of a sub-contractor’s project staff concerned with this 
! i project. 

An on-call member of a sub-contiactor ’s agency, performing on the project 
I j as needed. 

I 1 An advisor or consultant, not considered part of the regular project staff. 



A manager or specialist within th^ agency, concerned with similar matters 
across several projects of the agency. 



□ 


A member of an agency or unit that is participating 
portions of this project. 


in or cooperating with 


□ 


A member of an agency or institution that will be a 
project’s efforts or products. 


principal user of this 


□ 


Other : 








47. Project role: | H not fill in; for ADP use only) 
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Form OA 



Project \i 
Person H 



GENERAL POSITION ACTIVITIES 
QUESTIONNAIRE 



Name of Respondent: 



INSTRUCTIONS: 

Consider each of tlie following statements which may describe or 
characterize something that you do In your work cn this project. As 
you consider each item, you arn to proceed in two steps: 

First, consider whether the activity applies to your work. If 
the answer is NO, then it Is definitely not a part of your Job on the 
project (as when It Is delegated by you to a subordinate worker). 

Second, and only If the activity does apply for your work, you 
roust then decide how significant a part of your work it represents. 

In making this decision you are to consider and weigh its importance, 
frequency of occurrence, relevance, or any other factor which you think 
determines to what extent the activity contributes to your work. 

You are to allot a value between 0 and 7 to each item according 
to the following seal : 

U. Definitely not a part of my project activity, does not apply. 

1. Under unusual circumstances may be a minor part of my work. 

2 . 

3. 

4. A substantial part of my work. 

5 . 

b. 

7. A most significant part of my work. 

Thank you for your time and care in completing this form. 



This questionnaire was developed by 
Teaching Research, a division of the Oregon 
State System of Hlgh'ir Education, pursuant to 
a grant from the U.S. Office of Education. 




APPENDIX 1 (contM) 



F04-2 



lifitor a nutnber between 0 and 7 In the blank before eacli etatement, for general 
catogorieH .inci for more specific activities. 

ricase do not omit any listed Items. 

AfJSWKR EVERY ITEM. 

General Categories of Work: 

A. Reading . 

Designing or planning procedural activities for tiie project. 

^ Developing research tools or other information-gathering instruments. 

D. Collecting project data. 

E. Analyzing data. 

F. Writing. 

Supervising and coordinating actions of others, and/or of material 

resources. 

H. Teaching or training. 

I. Meeting, consulting, or advising. 

J- Other general category of work, (specify) 



K. Other general category of work, (specify) 



Specific Categories of Work: 

A. Reading. 

i. Reading recent project-related research. 

2. Reading scholarly essays. 

3. Reading methodological documents presenting information regarding 

methods of inquiry and/or analysis. 

A. Reading **in-house** materials and correspondence. 

5. Other reading: 



O 
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F04-3 



B. Designing or planning procedural activities for the project. 

1. Identifying relevant variables for considercition. 

2. Developing conceptual frameworks or general patterns of project design. 

3. Developing methodologies co be used in the project. 

4. Organizing a coherent program of activities. 

5. Designating sampling procedures. 

6. Designating general statistical treatment to Le used. 

7. Designing system models for computer application to data. 

8. Other designing: 



C. Developing research tods or other information-gathering instruments. 

1. Constructing questionnaires. 

___ 2. Developing test items. 

3. Developing interview outlines and schedules. 

4. Developing observational techniques. 

5. Other instrument developments; 



D. Collecting project data. 

1. Interviewing. 

2. Surveying literature. 

3. Conducting laboratory experiments. 

4. Administering questionnaires. 

5. Administering tests. 

6. Other data collection: 



E. Analyzing data. 

____ 1. Preparing or using frequency tallies and/or marginal distributions (as in 

Chi-Square teats). 

2. Computing or using measures of central tendency (i.e.f means, medians, 
modes, arithmetic average) . 

3. Computing or using correlation coefficients, including simple correlations 

analyses. 

4. Computing and interpreting simple tests of significance of differences in 
observed data (such as C^tests). 



APPENDIX 1 (cont'd) 



F04-A 



3. Computing and Interpreting data from analysis of variance designs, 

o. Computing and Interpreting regression analyses. 

7. Examining and Interpreting non-quantlfled Information (such as verbal 
responses, observed activities, etc.). 

8. Computing item analyses of test Items. 

9. Other data analysis: 



F. Writing. 



1. Writing correspondence. 

2. Writing research proposals. 

1. Writing niajor project reports. 

4. Writing Interim, status, or periodic reports. 

3. Writing for professional publications. 

6. Writing administrative reports. 

7. Writing literature surveys. 

8. Other writing: 



G. Supervising and coordinating actions of others, and/or of material resources. 

1. Procurement of project staff. 

2. Establishing contact with and participation by other personnel or agencies. 

3. Reviewing performance of project personnel. 

4. Communicating personnel evaluations to Individuals. 

3. Scheduling project activities. 

6. Allocating responsibilities to project personnel. 

7. Other supervision: 



H. Teaching 

1 . 

2 . 

3. 

4. 

3. 



or training. 

Participating In classroom Instruction. 
Participating In conduct of seminars or workshops. 
Providing on-the-job training to Individuals. 
Designing appropriate learning situations. 

Other Instruction-related activities: 



er|c 



4C2 
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!• Meeclng, consulclng, or advising. 



1. ConCacCs with funding sponsor or monitor of project. 

2. Contacts with higher agency management for review of project. 

Presentations made at professional meetings to communicate various 
aspects of project activities or results. 

4. Meeting with visiting personnel from other agencies. 

5. Other participation in meetings: 



J. Other general category of work: 

(specify and rate each specific activity you .:o) 

1 . 

2 . ^ 

3. 

4. 



Write in the specific activities you do in any other £,c.;hi3i category of work, and 
rate each. 




l5o 
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K. M j S('e 1 1 aneoiiK Activities. 

1. Edit in)' ,nni/or proofing.' nf printed mater iaJs. 

2. ForinulntinK iiypothescs or questions tn be answered by research, 

3. Determining constraints to problem solution, such as time, 

money, personnel, and market factors* 

4. Developing budgets for tasks or projects. 

3. Planning and/or making arrangements for field tests, training, 

trial centers, demonstratirms , installations, etc. 

6. Planning of behavioral, attitudlnal, and/or learning change in 

some target group • 

7. Identifying appropriate measures for events, variables, or other 

measurement concerns . 

8. Fabricating of physical items, svich as response recorders, 

stimulus presentation devices, room partitions or furniture, 
prototype devices, etc. 

9. Performing aspects of job and/or task analysis. 

10. Deriving or otherwise verifying the merit and/or relevance of 

student performance objectives (behavioral objectives). 



11. Collecting and organizing inforriation relevant to the preparation 
of a public infori.iation, dissemination, product distribution, 

or marketing plan. 

12. Drawing implications from the results of prior research (inter- 
pret, evaluate, and synthesize the relevant literature). 



13. Analyzing the nature of various audiences or ‘^publics*' to 
prepare appropriate communications . . 

14. Writing of computer programs for data handling or analysis. 



15. Writing of programmed instruction outlines and/or frames. 



16. Writing of detailed lesson plans. 



17. Conducting demonstrations of development products before various 
groups, and answering questions asked by members of the group. 

18. Preparing visual materials, such as films, slides, video tapes, 
visual teaching aids , etc . 

19. Conferring with colleagues, staff, and/or students. 



20. Interacting directly with personnel of other agencies, such as 
for field tests, at trial J^Qfjing centers, potential users of 
R & D products , etc . 



Speaking before public or specific target audiences. 



21 . 
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Form 06 



Pag e o f P ^ g 



Onsite Data Reduction 
Interview Recap: Standards 



Proj ect 
ID No, 



Interviewee 
ID No. 



Page 

No, 



Output 
No , 



Interviewer: 



^ .e/ ,'>• 

^ 4' 
V (y 






Clus ter Primary 

Category Category 



ABC 



D E F G H I 
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Form 07 



Page o f ages 



Onsite Data Reduction 
Interview Recap : Tasks 



Project Interviewee 



ID 


No. 




ID No. 
















N 


0 


P 




R 







Interviewer : 









Page 

No. 



Output 

No. 



^ 

<y Cluster Primary 

^ ^ Catepory Category 



ABC 



D E F G H 1 



Tasks 



O 

ERIC 



!o'; 
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J- 1 -/ o 
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Form 08 



P a ge o f P ages 



Project Interviewee 



ID 


No. 


ID No. 















Onsite Data Reduction 
Interview Recap : Enablers 



Interviewer: 




Enablers 

S T U V SI - Knowledges, S2 - Skills, S3 - Sensitivities 



. _ } 










i 

.. __ j 










1 

.. .. 1 










i 

1 

1 










( 

) 

i 










1 
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Source of Information: 



APPENDIX 1 (cont'd) 

Project Information from Project Officer 



Worksheet 02 



Project Officer No: 

I. Confirmation of Project Information 

Project Title : 



Principal Investigator; 



Setting; Location; 



Project Funding ; Total: Current Piacal Year ; 

Project Calendar Duration ; 



Current Phase of Project ; 



II. Project Abstract 

Purpose (s) of Project; 
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APPENDIX 1 (cont’d) 



W02-2 



Description of Producf to Produced; 



111. Project Staffing 

Principal Staff Members : 

Name: Title: FTE: 



Additional Staffing Info ; 




/>lPPENDIX 1 (cont'd) 



W02-3 



IV. InsCiCucional Linkages 
Agency Relationships ; 



V. Project Officer Judgenents 

Likelihood of Project Staff Cooperation ; 

Good Moderate Poor 

Comments : 






Officer Judgement or Project Appropriateneae to RPD&E ; 



VI. Additional Information 



APPENDIX 1 (cont'd) 



Worksheet 03 





Project Selection Data 


Project 

Focus 


Project 
ID // 




a) Title: 

b) Invest: 


R D D or 


E c) Location: 




d) Total: Funding Duration 


t 


e) Current: 



ERIC 


)/ 147 jL "X 



f 



APPENDIX 1 (cont'd) 



Worksheet 04 



I 



Project Identification Coding Sheet 



The following chart is used to Identify a new site by code number 
and includes reference to initial site information: 



1 Code 


j 

Project 


Site 


Project 


j Number 


Acronym i Full Project Title 


Location 


Director 


1 001 


. 1 

1 


[ 

1 

1 ; 




' 002 


1 i 

i 1 

t 1 

1 

t 1 


; 1 

1 

1 : 

1 

1 

i 


1 J 

i 

I 

! 



O 
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APPENDIX 1 (cont'd) 



Worksheet 05 



Project Perso’.mel Coding Chart 



The following chart is used to identify project personnel by code 
number : 



Project 
Code 
• Number 


Project 
Personnel 
Code Number 


Name of 

Project Personnel 


Project 

Code 

Number 


Project 
Personnel 
Code Number 


Name of 

Project Personnel 


001 


01 


Fred .Smith 


006 


05 


Gar Smith 


001 


02 


John Doe 


007 


01 


Jane Green 
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APPENDIX 1 (cont'd) 



Worksheet 06 



Data-Collection Personnel Coding Chart 



The following chart is used to identify the data collection agency 
personnel by code numbers: 





Code 

Number 


Name of Agency Personnel 






01 


Fred Smith 






02 


John Doe 






03 


Jane Jones 






• 
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APPENDIX 1 (cont'd) 



Worksheet 07 
Project It I 






TRAVEL INFORMATION 

ACCOMMODATIONS NEAR PROJECT LOCATION: 

Hotel/Motel: 

Address : 



Telephone: ( ) 



Room Extension; 



Hotel /Motel : 
Address : 



Telephone: ( ) 



Room Extension: 



TRANSPORTATION 

From Airport to Hotel: 



Costs : 



From Hotel to Agency: 



Costs ; 



From Hotel to Airport; 



Costs : 



O 
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APPENDIX 1 (cont’d) 



Worksheet 08 



DISSEMINATION BROCHURE 

Checklist of Materials to be Provided 
by RDDE Project Directors 



This checklist was developed by the 
Teaching Research Division of the 
Oregon State System of Higher 
Education persuant to a grant from 
the U. S. Office of Education 
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APPENDIX 1 (cont’d) 



Worksheet 08 
Allocated 

Project It lU 1 J 

DISSEMINATION BROCHURE 



Checklist of Materials to be Provided 
By RDDE Project Directors 



In an attempt to quickly gain as much understanding as possible 
of your project, the following documents and/or information is being 
requested of you. It is not necessary for you to expend much time 
in the preparation of this material. Simple line charts, handwritten 
statements, etc, are quite acceptable. As a guide for you to use in 
your preparation, a number of attachments have been included. Informa- 
tion collected is to be collated and presented in a form designed to 
describe RDD&E activities as they exist in the field, and to preclude 
any use of data for judgment purposes,^ 



O 

ERIC 



1. 


One copy of the original project proposal (inclusion 
of the budget statements is left to your discretion,; 


m 






2. 


One copy of each official project report. 

One copy of any available dissemination materials 
which describe the project (brochures, pamphlets, 
etc, ) , 




3. 


1 1 1 






4. 


An organization chart of the project which includes 
the title of each position, the names of personnel 




assigned to these positions and the percentage of 
their time assigned to the position, (See Attachment 
A for example) , 


n 1 






5. 


A general schedule of project events as ordered over 
time, (See Attachment B as an example). 


[ 1 1 






6 . 


Job descriptions of all project staff. (See Attachment 1 

C for an example). 


1 i 1 






7. 


( 

Vita for all project staff. The vita should include ^ 


_u 



the following : 

A, Highest degree awarded 

B, Area(s) of speciality 

C, Name of undergraduate institution 

D, Name of graduate institution(s) 

E, Sex 

F, Age 

G, Employment history 

H, Publications 

I, Salary level 




APPENDIX 1 (cont'd) 



8. Please indicate as accurately as possible the total 
number of professional staff members employed by 
your organization during fiscal years: 

1968/69 

1969/70 

1970/71 

9. Indicate as accurately as possible the yearly 
attrition rate for these personnel during: 

1968/69 

1969/70 

1970/71 

lOi What was the total level of financial support for 
the organization during: 

1968/69 

1969/70 

1970/71 



Comments : 



O 
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APPENDIX 1 (cont'd) 



W08-3 



ATTACHMENT A 

Project Organization Chart 
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APPENDIX 1 (cont’d) 



W08-4 



ATTACHMENT B 
Project Time Lines 




Establish objectives (1) 

Make team assignments (2) 

Design of evaluation instruments (3) 

Area specialists begin package 
development (4) 

First formative evaluation (5) 

First package re-design (6) 

Summative evaluation begins (7) 




its 

o 
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APPENDIX 1 (cont’d) 



W08-5 



ATTACHMENT C 
Job Descriptions 



John Doe - Project Director 

Responsible for coordinating all activities required for the success- 
ful completion of the project. 

In order to accomplish his goal, he is responsible for coordinating 
activities at several levels, He is responsible for coordinating the 
fiscal and reporting activities between participating institutions. He 
is also responsible for coordination of activities both between and with- 
in each of the projects’ phases. 

In addition, due to the special background and interest of this pro- 
ject director, he will also be directly involved in the planning, design 
and production of materials directly related to the skill of .... 

Fred Smith - Evaluation Director 

Will assist in translating specific enabling objectives into operational 
terms for measurement purposes. He will then identify and/or construct 
criterion instruments necessary for measuring the specific enabling 
objectives previously translated. He will be responsible for the testing 
procedures, and the analysis of data. 



i 
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APPENDIX 1 (cont'd) 



Worksheet 09 



ON 
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APPENDIX 1 (cont'd) 



Worksheet 1 L 



Contextual Cue Items’*' 



i. 



Do any of the following affect either the categories of tasks or 
the way in which tasks are performed on the observed proj cct ? (Use 
spaces below as soon after interview as possible 0 

1. Physical setting 



a. 


total project space 


: size , sound , color , smell 


b. 


individual working 


space 


c . 


project space 




d. 


climate , geographic 


: temperature , humidity 


e. 


other 





2. Management process and communication with project 

a. decision-making (consensus , democratic , authoritarian, etc . 

b. staff selection, retention, promotion 

c. staff meetings 

d. use of memos, charts, etc, 

e. availability of peers, supervisors 



3. Time, money or other factors which influence staff 



A. Interaction with consultants or other agencies 



5. Relationship of project to agency 



* This worksheet is prepared individaally by each interviewer, preferably 
with reference to each interview conducted. It may also be used as the 
basis for team debriefing, thus supplementing contextual data. 



O 
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APPENDIX 1 (contM) 



Wll-2 



(}. .Support .services - lielp or hindrance 



a . secretarial : prompt /slow, prof essional/ poor 

b . media: prompt /slow, quality, availability 

c. research and/or evaluation team or unit: help or hindrance 



7. Overall philosophy or culture of project or agency: do concepts of 

excellence, expedience, politics, or professional inter-action affect 
the content or nature of this project. 



8. Group process factors: friendship, professional enr -ties, group 

consolidation or fragmentation. 



9. Reporting Responsibilities: intra-agency reporting. Interim reports 

to funding agency, final reports. 



10. Sense of emotional financial or professional payoff; or sense of 
importance or worth of project goals. 



11. Any other factor you feel has influenced the content or value of 
data collected. 





APPENDIX i (contM) 



Worksheet 12 



Interview Record Form 
Standards 



I literv iewor 




Project 




Interviewee 

ID// 



Output 



] 


LD// 


1 







Focusing Leads and 
Follow-ap Probes 

”We would like to find out what 
yoi\ can tell us about the pro- 
duction of 

(output) . 

Opening tandards'* * inquiry 

’*Let's look first at the 
standards held for the 
quality of (this output). 

For (output), . , 

, . ,what standards were 
used to judge its 
adequacy?” by 
whom? 

• . .what did you use 
or make judgments 
abr .t?” 

, • .how about judg- 
ments about the 
processes used? 



Elaborate: Consensus 

Perfjonal feeling 
Meets criteria 



Check Notes: (To facilitate follow-up inquiry, re- 
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W12-2 



Inter'/iew Record Fon.; 
Tasks 



Interviewer 




Project laterviewet 

ID// ID// 

□□□ nn 



Output 

ID// 




O 
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Focusing Leads and 
Follow-up Probes 

Opening * * tasks** i nquiry 

"Now let’s look at the tasks 
necessary to (accomplish) this 
output to the standards set 
for it.** 

Probe 1; *'What steps were 
taken before getting into 
actual^^p reduction? As you 
spell these out, indicate 
which of them involved you 
personally. '* 

Elaborate: We planned 

We organized 
\Jc wrote objec- 
tives 

We designed 
We examined prob- 
lem 



Probe 2: **What steps were 

taken in producing " 

(output) . Indicate which 
ones of them involved you 
personally. 

Elaborate: We collected data 

We judged quality 
We distributed 
We built the thing 



Check Notes: (To facilitate follow-up inquiry, re- 




APPENDIX 1 (cont^d) 



W12-3 



Interview Record Form 
Enablers 



Interviewer 



□□ 



Proi act 
ID/, 






Interviewee 

ID// 



Focusing Leads and 
Follow-up Probes ^ 

Opening *'enabler'* inquiry 

'*I’d like now to ask about. . 



Probe 


i: . 


, .what must be 
known in order 

to . . . 




• 


. .which did you 
know?'* 


Probe 


2: . 


. .what skills mus 
be utilized?" 




• 


. .which did you 
use?" 


Probe 


3: . 


. .what sensitivi- 
ties were uti- 
lized?" 




• 


. .which did you 
use?" 



Output 

ID// 

□ID 



Check Notes: (To facilitate follow-up inquiry, re- 
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WorksliL’Ot I. J 



Tiainlng Suggestions Recap 



Page _ of ^ pa 



Project Interviewee Output 

ID// ID// ID// 

m m [XT 



Interviewer L 



Training Suggestions 



Associated Output, Task, or 



APPENDIX 1 (cont’d) 



Worksheet 14 



Interviewer Notes on Interrelationships 



Proi ect 
ID// 

□□□ 



Interviewer 

ID// 

□□ 



Note any observations about inter- 
relaticnships of personnel, part leu l.ar U* 
as they may clarify or suggest uni(|ue- 
nesses in operations not implied bv tin 
organizational structure . 



Notes 


Associated Personnel or Outputs 
(as relevant) 
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APPENDIX 1 (contM) 



WorkshoeL 16 



Project // n 
Interviewer 



Checklist of Interviewee Data Inputs 



The following information should be turned in to the Coordinator 
of New Data Inputs by the interview team members upon their return 
from their site visitation: 

1. A Form 03* for each interviewee (Job/Task Inventory) 

2. A Form 04* for each interviewee (General Position 

Activities Ou-'^stionnaire) 

3. Uncoded rec ^urms 06, 07, and 08 for each inter- 

viewee 

4. Forms 05-A ana 05-B from the Team Leader (Output 

Recap) 

5. A Worksheet 11 for each interviewee (Contextual 

Cue Items) 

6. A Worksheet 13 for each interviewee (Training 

Suggestions Recap) 

7. A Worksheet 14 from each interviewer (Interviewer * s 

Notes on Interrelationships) 

8. A final Output Map from the Team Leader 

9. A Contextual Map from the Team Leader 

10. A Project Organizational Structure from the Team 
Leader 



*Note: 



If these fo.ms are not completed before the last interview 
session, then the interviewer should make their completion 
the first order of business in the last interview. 



APPCNDIX 1 (cont*d) 
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Project 



(!hechlist of Project Datn Inputs 



Prior to the Interview team's site visitation, the following 
information should be turned in to the Coordinator of New Data 
Inputs by the: 



Coordinator for Site Selection 

1. A Form 01 (Project Contact Record) 

2. A Worksheet 01 (Project Selection Form) 

3. A Worksheet 02 (Project Information from 
Project Officer) 



Institutional Representative 

4. A Form 02 (Project Questionnaire) 

5. All items on Worksheet 08 (Dissemination Brochure) 

6. A Worksheet 09 (Output Interview) 

Team Leader 

7. An initial Output Map 



ERIC 
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APPENDIX 1 (cont*dJ 



Worksheet 18 



Chart of Project Information Status 



The following chart is used to record the retrieval of completed 
data input forms. Each space should be used as follows: 

1. After “project*', record the code number and acronym of the 
proj ect . 

2. After “Inst-Rep“, record the code number and name of the 
institutional representative. 

3. After “Prflwtr", record the code number and name of the 
profile writer. 

4. In space following the first “Onsite”, record the date that 
the institutional representative is to be onsite. 

5. After “Team onsite”, record the date that the interview team 
is to be onsite. 

6. Under “Site Personnel”, record the code number and the name 
of the interviewee across from the form number with which he 
is associated. 

7. Under “Agency Personnel”, record the code number and the name 
of the person in the data collection agency responsible for 
collecting that particular form from the identified person 
under "Site Personnel'*, 

8. If a form is in, check the “In” column. If it is out, check 
the “Out*' column, 

9. Under “Comment”, record anything pertinent to that particular 
data input. 

10. ”0M” means Output Map. 

11. “CP” means case-prof ile. 

12. ”IR“ means information on interrelationships. 

13. “POS” me;:ns Project Organizational Structure Chart. 

14. “CM” means Contextual Map. 

15. “UCR** means uncoded recaps. 

16. ”CRC*' means coded recaps. 



16 
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Chart of Project Information Statu.s 
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insi'-ui:p 

I’KKI.V.TK 



Onsite 
Tpam 0ns it f 



■onn 



0! 
02 
03 
0 3 
03 
03 
03 
03 
03 
03 

03 
1)3 
0/( 
0^1 

04 
04 
04 
04 
04 
04 
04 
04 
05A 
05B 

Wil 

Wli 

Wii 

Wll 

V>/li 

Wii 

Wii 

Wii 

Wii 

Wii 

Wi3 

VJ..3 

Wi3 

Wi3 

Wi3 

Wi3 

Wi3 

Wi3 

Wi3 

W13 



Site 

Personnel 



Agency 

Personnel 



Comment 



Form 



DCR 

CRC 



UCR 

CRC 



UCR 

CRC 



UCR 

CRC 



UCR 

CRC 



UCR 

CRC 



UCR 

CRC 



UCR 

CRC 



UCR 

CRC 



UCR 

CRC 



OM 

POS 

CM 

CP 

IR 

IR 

IR 

IR 

IR 

W08 

W08 

W08 

W08 

W08 

W08 

W08 

W08 
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Common t 



APPENDIX 1 (cont'd) 



Worksheet 20 



Profile Transmittal Sheet (Team) 



TO: 


DATE 


RE: 


FROM 



Attached is a copy of the first rough draft of case profile 

// . Please review it carefully, writing your comments directly 

on the draft and/or on a separate sheet. When reviewing this case 
profile please be particularly sensitive to the following: 

1. Accuracy of content and interpretations . 

2 . Completeness of obtained site inf omation and product 
descriptions . 

Please return the profile to me by ^ with 

your written comments. 



O 
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APPENDIX 1 (cont*d) 



Worksheet 21 



Profile Transnlttal Sheet (Coordinators) 



TO: 


DATE 


RE; 


FROM 



Attached is a copy of first rough draft of case profile il 

Please look it over carefully, writing your comments directly on 
the draft and/or on a separate sheet. When reviewing this case 
profile please be particularly sensitive to the following; 

1. Communicability of profile description for general audi- 
ences, including readability and clarity of style, 

2. Adequate accomplishment of the conceptual Intent of case 
profiles by PDD&E project. 

3. Adequacy of specific information provided for technical 
audiences and potential use in the future of others. 

4. Adequate handling of confidentiality of the information 
source. 

5. Adequate handling of the sensitivity of project infor- 
mation and manner of description. 

Please return the profi e to me by , with your 





-• r • 

i. c u 



written comments. 
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Worksheet 22 



Profile Transmittal Sheet (Editor) 



TO: 


DATE 


RE: 


FROM 



Attached is a copy of case profile // . Please look it over 

over carefully, writing your comments directly on the draft and/or 
on a separate sheet. When reviewing this case profile please be 
particularly sensitive to the follov/ing: 

ComF leteness of content . 

2. Forma t cons is t en cy as specified in the attached Section F, 
Attachment 1 from the "Manual on Data Management for RDD&E. 

3. Readability and clarity of style . 

4. Provision of definitions of technical terms , improper use 

jargon. 

5. Proper spell Ing , grammar , and syntax . 

b. Reasonable compatibility of format and style with the other 
profiles . 

7. Adequate handling of confidentiality and sensitivity . 

Please return the profile to me by , with-^our 
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written comnents. 



APPENDIX 1 (contM) 



U^orksheet 23 



Profile Transr/ttal Sheet (Project Director) 



Dear 



Enclosed is a draft copy of the case profile we have prepared for 
your project. To reiterate, the intent of this case profile is to provide 
in-depth information about selected outputs which comprise your 
project. We have not attempted to present a historically complete 
review of your project, but instead have only the snap-shot which time 
allowed. 

We would appreciate your review of the profile in terms of content 
accuracy , in terpretations , and editing of project documents contained 
within the profile. Comment may be provided directly on the profile 
and/or on a separate sheet. If you have any concerns regarding the 
profile, please feel free to call collect at OOO-OOO-OOOO for: 

Dr. John Doe, Project Director 
Dr. Fred Smit’n, Project Coordinator 
Mr. Gar Smith, Project Coordinator 



Because of the press of time, we need to receive word from you 
by . Enclosed is a releasr. form which we ask that you 

please sign and return. Also, please complete and return the attached 
rating sheet. A self-addressed envelope has been provided for the return 
of these forms and any other written comments. 
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W2 3-2 






'I'linnk you very much for your cooperation and npsist:ance in this 
mrtter and for your participation in the data colJection process. Y.,u 
might he interested to know, also, that most of the information has 
been c.oded fo'* computer storage--- available for a variety of summativc 
analyses across many projects. 

Thank yuu again for your assistance. 

Sincerely , 



John Doe 

Rfisearch Professor 



jj 
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W-23-3 



TEACHING RESEARCH 

A* Difiiioi: '■/ :!k Sluic SjUsm if EJucjtii/H 
MONMOUTH, OREGON 97361 



Telephone (503)8iS-12:o 



Permission is hereby granted for the inclusion of: 



within the outputs of a project for "The Generation of 
Information to Support Long-Term Manpower Studies of and 
Planning for Training Programs in Educational Research, 
Development, Diffusion, and Evaluation" which is being 
carried out under U.S. Office of Education Grant No. 
0-70-4977. 



Project Director: Date 



A CASE PROFILE OF 



A PROJECT TITLED: The Evaluation of the Early 

Childhood Education Program 



A PROJECT OF: 



Appalachia Educational 



Laboratory, Inc. 

P. 0. Box 1348 

Charleston, West Virginia 25325 



AS PREPARED BY: 



Teaching Research Division 

Oregon State System of Higher Education 

Monmouth, Oregon 97361 
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APPENDIX 1 (cont’d) 



W23-4 



Site Rating of ; 



(Case Profile Number 17) 



(Title of proj ect . ) 



To Director (and/or rater) : Following your review of the Case Profile prepared on 

the basis of information obtained from you and your staff, a rating of the 
representativeness of the Profile is requested. To assist in this rating, 
look at each item below and rate it on the basis of the key provided. Items 
(1), (2), and (3) are referenced throughout the profile, with primary concen- 
tration in Chapters I and II. Items (4), (5), and (6) indicate the location 
in the Profile of the specific items involved. Item (7) is a measure of the 
degree to which the Profile as a whole represents the project. 

Rating Schedule 

Check box A, B, or C as follows: 

A = Representative of a majority of operational concerns. 

B == Representative of only a part of operational concerns (concerns 
of significant proportions omitted.) 

C = Major concerns not covered. 



Focus of rating 


A 


B 


c 


1. Description of organizational 
structure 


□ 


n 


□ 


2. Description of organizational 

operations , interrelationships 


□ 


□ 


□ 


3, Description of the ends being 
sought by the project 


□ 


□ 


□ 


4. Representativeness of all 
outputs indexed (Ch. II) 


□ 


□ 


□ 


5. Representativeness of the 
outputs analyzed (Ch. Ill) 


□ 


□ 


□ 


s 

fi. Representative accuracy of the 
data on outputs (Appendix) 


□ 


□ 


□ 


7. Overall representativeness 


□ 


□ 


□ 



of the Profile 
General Comment, if any; 



Comment, if any 



APPENDIX 2 

Computer Codes: Project Characteristics (Form 02) 

Item //I Description and coding 

7. Total Duration of the Project. 

Level lA, Variable 11; Level IB, Variable 10 

1. 1-7 months 

2. 8-11 months 

3. 12 months 

4, 13-15 months 

5. 16-23 months 

6, 24-29 months 

7, 30-36 months 

8, 37 months or more 

8. Type of Contracting Organization in Which the Project Resides 

Level lA, Variable 12; Level IB, Variable 11 

1. Education Research and Development Center 

2. ' Regional Educational Laboratory 

3. College, School of Education, Psychology Dept., or other university 

office or agency 

4. Private Non-Profit Organization 

5. Industry or Commercial Organization 

6. Public School or School District 

7. State Department of Education 

8. Professional Association 

9. Agency of the Federal Government 

10. Other 

9. Project Funding from all Sources 

a. Total - Level lA, Variable 13; Level IB, Variable 12 

b. Current Fiscal Year - Level lA, Variable 14; Level IB, Variable 13 

c. Last Year - Level lA, Variable 15 

1. No funding 

2. $20,000 or less 

3. $20,001-$100,000 

4. $100,001-$225,000 

5. $225,00l-$500,000 

6. More than $500,000 

11. Primary Source of Funds 

Level lA, Variable 24; Level IB, Variable 14 

1. US Office of Education 

2. Other federal agency 

3. State 

4. Private foundation 

5. Local school district 

6. College or university 

7. Industry 

8. Other 
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Level lA, Variable 64; Level IB, Variable 34 



I- Single agency 

2. Single agency w/subcon tractors 

3. Two agencies cooperating 

4. Two agencies cooperating w/subcontractors 

5. More than two agencies cooperating 



18. Output 
Level 



Commitments 
lA, Variable 65 ; 



Level IB, Variable 35 



!• 1 Focal output 

2, 2 or 3 focal outputs 

3. 4 or more focal outputs 
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Computer Codes: Personnel Characteristics (Form 03) 
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APPENDIX 3 



Computer Codes: Personnel Characteristics (Form 03) 



4. Your functional job title, most descriptive of your work role on 
this project . 

Level 2A, Variable 4; Level 2B , Variable 4 
Not coded at present 



5 . Sex 

Level 2A, Variable - : Level 2B, Variable 5 

1 . Male 

2. Female 

6 . Age 

Level 2A, Variable 6; Level 2B, Variable 6 

1. Under 20 

2. 20-24 

3. 25-29 

4. 30-34 

5. 35-39 

6. 40-44 

7. 45-49 

8. 50-54 

9. Over 54 



7. Highest Degree you hold. 

Level 2A, Variable 7; Level 2B, Variable 7 



29. What level of academic training does this require? 
Level 2A, Variable 38 

1. Doctorate 

2. Specialist or Professional 

3. Masters 

4. Bachelor 

5. Associate 

6. Technical License 

7. None of these 

8. Other 

9. Number of professional publications authored. 

Level 2A, Variable 8; Level 2B, Variable 8 
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APPENDIX 3 (contM) 



i Uescr ip t i and c oding 



1 . none 

2. i-3 

3. A-8 

4. 9-15 

5. 16+ 

10. Major area of specialty for highest degree. 

Level 2A, Variable 9 & 10; Level 2B, Variables 9 & 10 



11. Minor areas of specialty for college study. 
Level 2A, Variables 11, 12, 13 

1 . Education/ teaching 

2. Education administration 

3. Educational research 

4 . Guidance/counseling 

5. Vocational training 

6 . Statistics /measurement 

7. Psychology 

8. Engineering 

9. Computer sciences 

10 . English/writing 

11. Business administration 

12. Other 



12. Your annual salary. 

Level 2A, Variable 14; Level 2B, Variable 11 

1. Under $5,000 

2. $5,000-$8,999 

3. $9,000-$ll,999 

4. $12.000-$16,999 

5. $17,000-$21,999 

6. $22,000-$29,000 

7. Over $29,000 



13. National Professional Memberships 
Level 2A, Variable 15-21 



31. Please check all support services from the list below, provided by 
other persons or agencies, that you must have to carry out your job 
on this proj ect • 

Level 2A, Variables 40-62 



32. Please check all the support equipment listed below that are 
immediately available and used by you on this project. 

Level 2A, Variables 65-75 
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1» If checked 



APPENDIX 3 (rent’d) 



Ite m /M lJ e.se ri p t ion and jJotUnju: 

3.3. Wluit amount of travel is expected of you on this project? 

Level 2A, Variable 78 

a. Number of trips per year 

1. none 

2. 1-2 

3. 3-5 

^4. 6-10 

5. 11-15 

6. 16-25 

7. More than 25 
Level 2A, Variable 79 

b. Average duration of each trip 

1 . no trip : involved 

2 . less than 1 full day 

3. 1 day each trip 

4. 2 days 

5. 3 days 

6. 4-5 days 

7. 6-8 days 

8. 9-15 days 

9. more than 15 days 

34. To what extent are project manpower resources (number and capa- 
bilities) adequate? 

Level 2A, Variable 80 



35. To what extent are project time lines and available period of 
performance adequate? 

Level 2A, Variable 81 



36. To what extent are project financial resources adequate? 

Level 2A, Variable 82 

1. Reasonably adequate 

2. A bit tight 

3. Extremely short 

3/* Is some reasonable amount of project time made available to projec 
personnel for each of the following? 

Level 2A, Variables 83-90 



Does your project or agency expect you to engage in the following? 
Level 2k, Variables 91-98 
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APPENDIX 3 (cont’d) 



) U'lii i r t itui .mii 

39. Check ail uf Lhe lollowing which are usually provided to you by the 
project leader and/or his key subordinates. 

Level 2A, Variable 99-107 

1 . if yes 

2. if no 

40. Project structure as viewed by project personnel: 

Level 2A, Variable 109; Level 2B, Vari-'^ble 27 

1. Pyramid 

^ 2. Corporate structure 

3. Inverted pyramid 

4. Chain of command 

5. Leader less 

6. Other 

41. To what extent is coordination required with this project? 

Level 2A, Variable 110; Level 2B, Variable 28 

42. To what extent does this project require coordination with other 
agencies? 

Level 2A, Variable 111; Level 2B, Variable 29 

1. Very minimal coordination 

2 . 

3. 

4- Moderate amounts of coordination 

5. 

6, 

7. Very extensive and involved coordination 

46, What is your relationship to the primary contractual project? 

Level 2A, Variable 129; Level 2B, Variable 41 

Regularly a part of the prime project staff. 

An on-call member of the prime agency, performing on the project 
as needed . 

Regularly a part of a sub-contractor’s project staff concerned 
with this project. 

An on-call member of a sub-contractor’s agency, performing on 
the project as needed . 

An advisor or consultant, not considered part of the regular 
project staff. 

A manager or specialist within the agency, concerned with similar 
matters across several projects of the agency. 

A member of an agency or unit that is participating in or 
cooperating with portions of this project. 

A member of an agency or institution that will be a principal 
user of this project's efforts or products. 

Other 
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1 . 

2 . 

3. 

4. 

5. 
b. 

7. 

8 . 





APPENDIX 4 

Data Sets and Categories 



NOTE: Code numbers assigned categories were assigned in sequence for 

each Category Set. As a function of the evolutionary nature of the 
developmental process used in creating the Category Sets, occasional 
numbers were dropped from use and the items conntected to them integrated 
into other (or new) and more useful categories. The reader or user, 
therefore, should not be concerned about missing numbers in any apparent 
sequence. — -- — e— ^ 



202 



APPENDIX 4 



r 


Data Sets and Categories 
Surmnary of Category Sets 


Category 

Set 


Category Set Label 


A 


Structures of Outputs 


B 


Levels of Outputs 


C 


Charac ters of Outputs 


a 


Functions of Outputs 


DE 


Output Clusters 


FGHI 


Primary Categories of Outputs 


J 


S true tures of Standards 


K 


Self/Others Report (for Standards) 


LM 


Primary Categories of Standards 


NO 


Task Clusters 


P 


Self/Others Report (for Tasks) 


QR 


Primary Categories of Tasks 


S 


S true tures of Enablers 


T 


Self/Others Report (for Enablers) 


UV 


Primary Categories of Enablers 


w 

X 


Output Completion Stage 
l^rojeca Role of Interviewee 


Y 


Project Focus 


ERIC 


Character of Supported Focal Output 

; : 203 O 
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AFPENDIX 4 (cont'd.) 

Category Set A 
Structure of Outputs 



A 



Set A 
Coding* 


Structure 


1 


Product 


2 


Event 


3 


Condition 



* This code is to be given each output listed for a project prior to site 
visitation by an interview team, and modified as appropriate on site. It 
is recorded in Column A on Form 05-B for each output listed, and in Cell A 
on I’orms 06, 07, and 08 for each output to be analyzed. Responsibility for 
final assignment of this code rests with the interviewer. 



B 



APPENDIX 4 (cont’d.) 



B 



Category Set B 
Level of Outputs 



Set B 
Coding * 

1 

2 

3 



Level 

Focal 

Component 

Facilitating 



* This code is to te given to each output listed for a project prior to 
site visitation by an interview team, modified as appropriate on site and 
recorded in Column B on Form 5-B and in Cell B of Forms 06, 07, 08 for 
each output to be analyzed. Responsibility for final assignment of this 
code rests with the Team leader. 
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APPEND LX 4 (cont’d.) c 

Category Set C 




Character of Outputs 


/ 




Set C 
Coding* 


Characters 


1 


Knowledge 


2 


Technology 


> 3 


Implementation 




Information 



* This code is to be given each output listed for a project prior to site 
visitation by an interview teanti modified as appropriate on site, and re- 
corded in Column C on Form 05-B for each output listed, and in Cell C on 
Forms 06, 07, and 08 for each output to be analyzed. Responsibility for 
assignment of this code rests with the Coding Resolution Tear.'. 



o 

ERIC 


2^:209 2:0 



a 



APPENDIX 4 (cont’d.) 



a 




Category Set a 
Function of Outputs 



Set a 
Coding * 

1 



Function 

Policy-Setting 

Management 

Production 



I 




* Responsibility for asr^ignment of this code rests with the interview 
team leader with confirmation by Coding Resolution Team, and entry Is to 
be made in Column a on Form 05-B. 
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DE (A-1) 



APPENDIX 4 (cont'd.) 



DE(A-l) 



Clusters of Outputs 

Clusters of Products (Category Set A-1) 



Set DE 
Coding* 


Clusters 


01 


Reports /Contracts 


02 


Data Ins t ruments /Tediniques 


03 


Guides /Manuals 


04 


Work Specif icai ions/ ‘Procedures 


05 


Work Management Systems 


06 


Data Management Sys terns 


07 


Resource Lists 


08 


Plans /Designs /Theories /Models 


09 


Equipment /Tools /Facilities 


10 


Publications 


11 


Promotional Materials 


12 


Instructnl Units/ln-School 


13 


Instructnl Units/Extra-School 


14 


Instructnl Units/Use w/Staff 


27 


Data (including printouts) 


28 


Taxonomies 


29 


Recommendations 


36 


Curricular Objectives 


37 


Critiques 


42 


Prediction Variables 



* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. It is recorded in Columns DE on Form 05B for each 
product listed for a project, and in Cells DE on Forms 06, 07, and 08 
for each product to be analyzed. 



i)K (A-2) 



APPENDIX 4 (cont’d.) 
Clusters of Outputs (cont*d.) 



DE (A-2) 



C lusters of Events (Category Set /-2) 



Set DE 

Coding* Clusters 

15 Staff Meetings 

16 Conferences/Seminars 

17 Workshops /Institutes 

18 Presentation 

19 Benchmark Decisions 



30 Committee/Consultant Meet ins 

31 Field Trails /Experiments 

32 Recruiting/Placing Staff 

33 Collecting/Analyzing Data 

38 Training 

39 Updating of Machine Systems 

40 Acquiring Supplies /Facilities 

43 Establishing Opns Parameters 

47 Coordinating Efforts 

48 Maintaining Accountabilit/ 

49 Establishing Field Sites 

50 Disseminating Info/Outputs 

51 Acquiring Funds 

52 Adapting Materials 



* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. It is recorded in Column DE on Form 05B for each event 
listed for a project, and in Cells De on Forms 06, 07, and 08 for each 
event to be analyzed. 










DE (A- 3) 



APPENDIX 4 (cont'd.) 
Clusters of Outputs (cont'd.) 



DE (A- 3) 



Clusters of Conditions (Category Set A-3) 
Set De 

Coding * Clusters 

20 Cooperative Relationships 

21 Working Environment /Atmosphere 

22 Quality Assurance 

23 Accountability 

25 Resource Accessibility 

26 Available Field Sites 

34 Coordinated Efforts 

35 Acceptance/Adoption of Outputs 

41 Effective Communication 

44 Provision for Services 

45 Informed Public 

46 Trained Staff 



* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. It is recorded in Columns DE on Form 05B for each 
condition listed for a project, and in Cells DE on Forms 06, 07, and 
08 for each condition to be analyzed. 
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FGHI (A-1) 



APPENDIX 4 (cont'd.) 



FGHI (A-1) 



Category Set FGHI 



Primary Categories of Outputs 



Primary Cateogires of Products (Category Set A-1) 
Set FGHI 

Coding* Primary Categories (A-1) 



0001 

0002 

0003 

0006 

0007 



Prototype Tryout Data 
Research Design 
Microformed Bibliographies 
Terminal Report 
Proposal 



0013 

0016 

0017 

0018 
0019 



Project Time Line/Schedule 
Advisory Committee 
Terminal Instr Objectives 
Enabling Instr Objectives 
Content Outline, Instructional 



0020 

0021 

0022 

0023 

0024 



Syndactic Text Instr Materials 
Remedial Units for Syndactic Texts 
Single-Concept Instr Films 
Video Tape Instr Modules 
Audio Tape Instr Modules 



U025 

0026 

0027 

0028 
0029 



Activities Outline: Instr Units 

Computer Asst/Ext Instr Units 
Linear Programmed Instr Text 
Intrinsically Programmed Text 
Site Assessment Criteria 



0030 

0031 

0032 

0033 

0034 



Materials Presentation Design 
Criterion-Based Test Item Pool 
Norm-Referenced Test Items 
Learner Data File 
Instructional Unit Use Guide 



0035 

0036 

0037 

0038 

0039 



Parent Instr Unit Use Guide 
Model: Developing Instr Units 

Computer-Based Instr Unit 
Research Data 

Interview Training Procedures 



0040 

0041 

0042 

0043 

0044 



Description of Program/Project 
Site Visit Checklist 
Selection Criteria: < Sites etc 
Guidelines for Site Visits 
Interview Data/Information 




0045 

0046 

0047 
0049 



Site Visitation Schedule 
List of Field Settings 
A/V Descriptions of Program 
Progress Check Test 




KGHI (A-1) 


APPENDIX 4 (cont'd.) FGHI (A-1) 

Primary Categories of Outputs (cont'd.) 


Set FGHI 
Coding * 


Prima»-y Categories (A-1, cont'd. ) 


0050 

0052 

0053 
0055 
0060 


Structural Communication Unit 
Panel Book Visuals : Audio Tape 

Abstracted Descript: Programs 

Report Specifications 
Validation Criteria. for Data 


0061 

0062 

0064 

0065 

0066 


Observation System 
Team Rotation Plan 
Cost Estimates 

Test Procedure, I-M Handling System 
List of Project Products 


0067 

0068 

0069 

0070 

0071 


Implementing Plan, I-M Handling 
Review/Procedures I-M Handling 
Design Review Data, I-M System 
Design for I^ Handling System 
Review of Des.lgn Report 


0072 

0073 

0074 

0075 

0076 


Specs : Hardware Requirements 

Specs: Software Requirements 

Specs: Operation Requirements 

Manual for I^ Handling Sys 
Specs: I-M Handling Sub- Systems 


0077 

0078 

0079 
0081 
0082 


Specs: Automated I-M System 

Major Task Descriptions 
Cost Categories: Oper System 

Interview Guide 
Questionnaire 


0083 

0085 

0086 

0087 

0088 


PERT Chart 

Project Operational Plan 
Bibliographies 

List of Potential Output Users 
List of User Needs 


0089 

0090 

0091 

0092 

0093 


Status Condition Report 
Report of Operations Analysis 
Performance Specs: I-M System 

Design Alternates for Evaluation 
Design Concepts, I-M System 


0094 

0095 

0096 

0097 
0099 


Automated I^ Handling System 
Description/Specs for Output 
Evaluation (Checklist 
Journal Article, Published 
Progress Report 


O 

ERIC 





FGHI (A-1) 



APPENDIX M (cont'd.) 





Primary Categories of Outputs (cont'd. 


Set FGHI 
Coding* 


Primary Categories (A-1, cont'd.) 


0104 

0105 

0106 

0107 

0108 


Instructional Package, Medium 
Instructional Package, Small 
Cumulative Post Test 
Master Tutor Instr Unit 
Supplementary Instr Materials 


0110 

0115 

0117 

0118 
0121 


Content Map for Instr Course 
Project Products Dstrb System 
Program (within project) Plan 
Formulated Policy 
S^-ript, Film 


0125 

0128 

0129 

0130 

0131 


Promotional Film 

Multimedia Leadership Course 

Television Program 

Animated Film: REading Program 

Live Action Film: Reading Prog 


0132 

0133 

0134 

0135 

0136 


Video Tape, TV Prog Component 
Serialized TV Adventure Script 
Serialized Storey Manuscript 
Television Program Magazine 
Curriculum for Reading Program 


0137 

0138 

0139 

0140 

0141 


Ref Manual: Instr in Reading 
Story Boards 

Non-fiction Books (commercial) 
Promotional Phonograph Records 
Film Strip Promotional Unit 


0144 

0145 

0146 

0147 

0148 


Financial Report 
Intraagency Opns Control Form 
Contracts for Services 
Live-Action Film for TV Show 
Animated Film for TV Show 


0149 

0150 

0151 

0152 

0153 


Ref Index: Film for TV Program 

Scripts for TV Program 
Curriculum for Ed TV Program 
Video Tapes, completed TV Prog 
Specifications for Facilities 


0154 

0155 

0157 

0158 

0159 


Specifications for Personnel 
Info/Idea Notebook for Writers 
Sound Track: A-V Presentation 

Evaluation Report 
Project Newsletter 



FCllI (A-1) 



APPENDIX 4 (cont’d.) 

Primary Categories of Outputs (cont'd.) 



FGllI (A-1) 



Set FGHI 

1 Coding* 


Primary Categories (A4l, cont'd. ) 


* 0160 

X 0161 

^ 0162 

0163 

0164 


Training Film for Educatjrs 
Diffusion Staff Trng Program 
Demo of Follow-Op to A-V Learning 
Training Film for Parents 
Diffusion Mannual/Guidelines 


0167 

0168 

0169 

0170 

0171 


Community Resource Directory 
Demographic Data 
Diffusion Impact Data 
Community Reiiource Data 
Pre-school Guide Activity Data 


0172 

' 0173 

0174 

0175 
0177 


Pre-sch Lrng Matter Teacher Use 
Pre-sch Lrng Matter Parent Use 
Pre-sch Lrng Activities > Tchr 
Pre-sch Lrng Activities, Parent 
Trng Program for Lay Personnel 


0180 

0181 

0182 

0183 

0184 


Pilot Instr Program (movie) 
Interagency Opns Control Form 
List: Dlstrlb Agencies/Points 

List of Resource Personnel 
Mailing Lists 


0185 

0186 

0187 

0188 
0189 


Needs Identification Data 
Specs: Free Product Dtbg Sites 

Descriptive Data Tests, subject 
Normative Data 
Data on Product Appeal 


0190 

0192 

0194 

0195 
0197 


Product Review/Critique 
Specs: Data Gathering Instrmnt 

Learner Achievement Data 
Social Interaction Data 
Descr of Research Variables 


0198 

0199 
0201 
0202 
0203 


Sight Vocabulary List 
Item Analysis Data 
Professional Report/Monograph 
Catalog of Available Products 
Learning Activity Books 


0204 

0205 

0206 
0208 
0209 


Learning Activity Toys/Games 
Pre-school Text Books 
Educational Greeting Cards 
Plans: Future Project Dvlpmnt 

Speech Texts 


O 

ERIC 
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APPENDIX 4 (cont'd.) 

Primary Categories of Outputs (cont'd.) 



FGHI (A-1) 



Set FGHI 
Coding* 


Primary Categories (A-1, cont 'd . ) 


0211 

0212 

0213 

0214 

0215 


Handling Sys: Project Outputs 

Educational Calendar 
Children’s Activity Center 
Satellite Management Center 
Learning Kit , commercial 


0216 

0217 

0218 

0219 

0220 


Dissertation 
Observation Record Form 
Quality Control Schema 
Student Performance Objectives 
Data Analysis Procedures 


0221 

0222 

0223 

0224 

0225 


Source List for Objectives 
Data Collection Procedures 
Taxonomy for Learner Outcomes 
Attitude Meas» Congitive Value 
Rationale for Data Instrument 


0226 

0227 

0228 

0229 

0230 


Rational for Data Instrument 
Compendium: Instr Objvs and Msmt 

Compendium; Instr Objectives 
Attitude Measure, Tolerance 
Technical Report 


0231 

0232 

0233 

0235 

0236 


List: Acquisition/Data 

Specs : Output Revisions 

Curricular Recommendations 
Baseline Data 

Single Concept Learning Unit 


0237 

0238 

0239 

0240 
0242 


Model of Skills Acquisition 
Model of Content Discipline 
Compendium of Measures 
Working Papers 
Data Handling Codes 


(244 

0245 

0249 

0250 

0251 


Coded Data 
Computer Printouts 
Staff Development Trng Unit 
Pupil Svc Personnel Trng Unit 
School Paper 


0252 

0253 

0255 

0256 

0257 


Information Brochures 
Curriculum Dvlpmnt Guidelines 
Interview Reports 
Field Test Plan 
Conference/Workshop Agenda 
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APPENDIX 4 (cont ’d.) 

Primary Categories of Outputs (cont'd.) 


Set FGHI 
Cod i.ng* 


Primary Categories (A-1, cont'd.) 


0259 

0260 
0261 
0266 
0267 


Evaluation Model 

Summative Evaluation Data 

Fonnative/Field Test Eval Data 

Journal 

Indexes 


0268 

0269 

0270 

0272 

0273 


Microform Copy 
Correspondence 

Computer Program Documentation 
Publication Proof Copy 
Publication (not identified) 


0274 

0275 

0276 

0278 

0279 


Usage Report 
Project Objectives 
Articles: House Publications 

Prospectus/Letters of Intent 
Conceptual Framework 


0280 

0281 

0282 

0285 

0286 


Specs: Filming Techniques 

Evaluation Plan/Design 
Memos/Directives /Bulletins 
Organizational Chart 
Survey Reports 


0287 

0238 

0289 

0290 

0291 


Key Indicators: Condition School 

Task/Time Allotments 
Directories 
Census Data 

Management Inf ^r nation Center 


0292 

0292 

0294 

0297 

0301 


Mediated Data Display 
Review/Refinement Procedures 
Operations Progress Chart 
Feedback System Classroom Mgmt 
PPBS Manual 


0303 

0304 

0305 

0306 

0307 


Computer Instr in Computer Use 
Computer Simulation: Operation 

Project Promo Presentation Pkg 
Clearinghouse: RDD&E Outputs 

Use Guide for Card Sort System 


0308 

0309 

0310 

0320 

0321 


Data-Handllng Cards 
Worksheets: Proj Docunentation 

Script: Film/Tape Presentation 

Trng Program in Group Dynamics 
Content Outline Guides /Manuals 
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FGHI (A-1) 



APPENDIX 4 (cont'd.) 



pnid (A-1) 



Primary Categories of Outputs (cont'd.) 



Set FGHI 




Coding* 


Primary Categories (A-.l, cont'd.) 


0324 


Budget Allocations 


0326 


Reporting Procedures 


0327 


Coordination/Liaisoa Procedure 


0329 


Work Assignment Record 


0332 


Magazine/Newspaper Article 


0334 


Theory Papers 



* Responsibility for assignment of this code rests with the Coding 
Resolution Team. It is recorded in Columns FGHI on Form 05B for 
each produc t listed for a project and in Cells FGHI on Forms 06, 
07, and 08 for each product to be analyzed. 
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FGHI (A-2) APPENDIX 4 (cont'd.) 

Primary Categories of Outputs (cont'd.) 
Primary Categories of Events (Category Set A-2) 

Set FGHI 



Coding* 


Primary Categories (A-2) 


0003 


Evaluating Hardware 


0011 


Acquiring Microform Hardware 


0012 


Assigning Work 


0014 


Disseminating Information 


0015 


Making Management Decisions 


0098 


Info Dissemination Workshop 


0103 


Acquiring Course Materials 


0114 


Field Tests 


0119 


Speeches 


0120 


Acquiring Funds 


0143 


Acquiring TV Studio Hardware 


0156 


Pre-use Critique: A-V Program 


0165 


Acquiring Facilities 


0176 


Training Workshop/Conf erence 


0178 


Acquiring Television Sets 


0191 


RDD&E on the Job Training 


0200 


Policy Settins; Seminars 


0234 


Reviewing Status/Quality 


0241 


Selecting Personnel 


0243 


Interviewing 


0246 


Interpreting Data Analysis 


0258 


Development (work) Conference 


0262 


Selecting of Performance Items 


0264 


Staff Meetings 


0265 


Collecting Data 


0271 


Modifying Systems 


0277 


Acquiring of Filming Equipment 


0284 


Staff Retreat 


0296 


Planning Proj Opns/Guidelines 


0311 


Surveying by Telephone 


0312 


Surveying Costs 


0313 


Selecting Field Test Sample 


0314 


Coding Data 


0316 


Coordinating Data Collection 


0317 


Maintaining Project Progress 



FGHI (A-2) 



FGHI (A-2) 



APPENDIX A (cont'd.) 



FGHI (A-2) 





Primary Categories of Outputs (cont'd.) 


Set FGHI 
Coding* 


Primary Categories (A-2 cont'd.) 


0318 

0322 

0323 
0325 
0328 


Maintaing Fiscal Acetability 
Negotiating Participation/Coop 
Making Policy Decisions 
Acquiring Operational Supplies 
Conducting Experiments 


0331 

0333 

0335 


Expanding Product Availability 
Surveying the Literature 
Revising Existing Materials 



* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. It is recorded in Columns FGHI on Form 05B for each 
event listed for a project, and in Cells FGHI on Forms 06, 07, and 08 
for each event to be analyzed. 






FGHI (A-3) APPENDIX 4 (cont'd.) 

I'rimary Categories of Outputs (cont'd.) 
Primary Categories of Conditions (Category Set A-3) 



FCHI (A-3) 






Set FCHI 
Coding* 

0004 

0005 

0009 

0010 
0054 

0056 

0057 

0058 

0059 
0084 

0100 

0101 

0109 

0111 

0112 

0113 

0116 

0124 

0126 

0127 

0166 

0179 

0193 

0196 

0207 

0247 

0254 

0263 

0283 

0295 

0298 

0299 

0300 
0302 
0315 



Primary Categories (A-3) 

Project Publicity 
Adequate Field Setting 
Managed Money Resources 
Adequate Staff 
Staff Awareness/Acceptance 

Compatible Work/Expense Rates 
Decision Making Structure 
Quality Output 

Operations Efficiency, Project 
Interagency Coordination 

Staff Welfare 

Intra/Inter Agency Relations 
Staff Morale 

Inter/Intra Agency Cooperation 
Project Accountability 

Site Personnel Welfare 
Managed Production Personnel 
Atmosphere: Feeling Effective 

Relationship with TV Networks 
Adherence to Project Philosophy 

Facilitated Product Distrib 
Consulting Services 
Intra/agency Coordination 
Effective Communications 
Project Management Structure 

Adequate Facilities/Equipment 
Community/Parent Involvement 
Monitored Project Operations 
Adequate Information Flow 
Healthy Atmosphere: Criticism 

Maintained Program Operations 
Test Construction Service 
Measurement Service 
Informed Public 
Project Leadership 



O 

ERIC 
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FGHI (A-3) APPENDIX 4 (cont'd.) FGHI (A-3) 





Primary Categories of Outputs (cont’d.) 


Set FGHI 
Coding* 


Primary Categories (A-3, cont'd.) 


0319 

0330 

0336 


Adequate Community Interface 
Adequate Support Linkages 
Coordinated/Compatible Outputs 



* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. It is recorded in Columns FGHI on Form 05B for each 
condition listed for a project, and in Cells FGHI on Forms 06, 07, and 
08 for each condition analyzed. 
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J 



APPENDIX 4 (cont'd.) 
Categoi 7 Set J 
Structure of Standards 



J 



I 






Set J 
Coding* 

1 

2 



Structure 

Output Standard 

Process /Ope rat ions Standard 






* Responsibility for assignment of this code rests with the Coding 
Resolution Team. It Is recorded In Column J of Form 06 for each state- 
ment of a standard associated with the output being analyzed. 
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APPENDIX 4 (cont'd.) 



K 









Category Set K 

Self/Others Report (for Standards) 

Set K 

Coding* Self/Others Report (for Standards) 

1 Self 

2 Others 

3 Team 



F 



I 




* Responsibility for assignment of this code rests with the interviewer 
as each interview is recapped on Form 06. As each statement of a stand- 
ard is recorded, the appropriate Self /Others code is entered in Column K. 
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LM (J-1) APPENDIX 4 (cont'd.) 

Category Set LM 

Primary Categories of Standards 
Primary Categories of Output Standards (Category Set J- 11 

Primary Categories (J- 1) 



LM (J-1) 



Set LM 
Coding* 



01 

02 

03 

04 

05 

06 

07 

08 

09 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 



Completeness of content 
Quantity of outputs/data 
Quantity of effort expended 
Communication and clarity 
Utility or value 

Acceptance by users 
Personal satisfaction/feeling 
Agreement /concurrence w/others 
Lack of errors/discrepancies 
Obvious (direct) termination 

Appropriate design/content 
Goal attainment 

Acceptance by others (in proj) 
Acceptance by sponsor 
Compliance w/sponsor guideline 

Compares favorably 
Internally consistent 
Satisfactory appearance 
Logical criteria 
Performs consistently 

Sources of variance controlled 
Functions as planned 
Successfully constrains/guides 
Terminology appropriate 
Awareness that outputs exist 

Components are complimentary 
Good physical repair/quality 
Operable by others 
Meets design expectations 
Lack of negative feedback 

Meets legal conutraints 
Created by reputable producer 
Inclusions are representative 



* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. It is recorded in Columns LM on Form 06 for each state* 
ment of ‘an output standard associated with the output being analyzed. 
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IJ^ (J-2) 



APPENDIX 4 (contid.) 



I>1 (J-2) 



rrimary Categories of Standards (coat'd.) 



Primary Categories of Process/Operations Standards (Category Set J-2) 



Set LM 

Coding* Primary Categories (J-2) 



01 

02 

03 

04 

05 



Personnel cooperate 
Personnel are satisfied 
Minimum correction required 
Deadlines are met 
Acceptable level of output 



06 

07 

08 

09 

10 



Work structure is efficient 
An expected activity occurs 
Staff contributions accepted 
Outside contributions accepted 
Maximum possible participation 



11 

12 

13 

14 

15 



Costs consistent w/estlmates 

No obvious omissions 

Work conducted w/ in budget 

No felt deficiencies 

Tasks perceived and acted upon 



16 

17 

18 

19 

20 



External cooperation gained 
External enthusiasm evident 
Desired personnel obtained 
Adequate reputation w/sponsor 
Performance respected 



21 

22 

23 

24 

25 



Follow-on proposals are funded 
Feedback occurs 
Project view accepted 
Costs acceptable for benefits 
Staff reflect trust 



26 

27 

28 

29 

30 



Values and objectives match 
Decisions result in action 
Closure reached on questions 
Creativity in work evidenced 
Personnel loss not excessive 



31 

32 

33 

34 

35 



Overtime worked voluntarily 
Resources available on request 
Personal growth/productivity 
Impact of effort favorable 
Outputs distributed/requested 



36 

37 

38 

39 

40 



Employment criteria met 
Staff adaptive to situations 
Equity in opportunity to input 
Evidence of pre-planning 
Outputs published externally 



LM (J-2) 


APPENDIX 4 (cont'd.) LM (J-2) 

Primary Categories of Standards (cont'd.) 


Set LM 
Coding* 


Primary Categories (J-2, cont'd.) 


41 

42 

43 

44 

45 


Guidelines are followed 
Revision yields Improvement 
Policy constraints minimal 
Responsibility taken willingly 
Resources used efficiently 


46 


Staff are self-directing 



* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. It is recorded in Columns LM on Form 06 for each state 
ment of a process/operations standard associated with the output being 
analyzed . 



NO 



NO 



Set NO 
Coding* 

01 

02 

03 

04 

05 

06 
07 



APPENDIX 4 (cont'd.) 
Category Set NO 
Task Clusters 



Task Clusters 



Clarifying problem addressed 
Formulating objectives 
Designing the output 
Producing the output 
Collecting/processing data 

Assessing the output quality 
Diffusing the out mt 



21 


Procuring professional staff 


22 


Effecting accountability 


23 


Procuring systems/services 


24 


Effecting quality control 


25 


Maintaining job satisfaction 


26 


Facilitating growth of staff 


27 


Enhancing physical environment 


28 


Maintaining equity among staff 


29 


Facilitating relationships 


30 


Effecting info flow patterns 


31 


Diffusing info within project 


32 


Diffusing info beyond project 


33 


Effecting decision mechanisms 




* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. It is recorded in Columns NO of Form 07 in conjunction 
with a "QR" classification for each statement of a task associated with 
the output being analyzed. (See Category Set QR) 
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APPENDIX 4 (cont'd.) 



P 



Set P 
Coding* 

1 

2 

3 



Category Set P 

Self /others Report (for Tasks) 

Self /others Report (for Tasks) 

Self 

Others 

Team 



* Responsibility for the assignment of this code rests with the inter- 
viewer as each interview is recapped on Form 07. As each statement of 
a task is recorded, the appropriate Self/Others code is entered in 
Column P. 
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QR (NO-01) 



QR (NO-01) 



APPENDIX 4 (cont'd.) 



Category Set QR 



Primary Categories of Tasks 



Set 




NO 


QR 




Coding* 


Primary Categories (Cluster NO-01) 


01 


01 


Review problem-relevant literature/info 


01 


02 


Specify context/scope/limits of effort 


01 


03 


Specify variables to be studied 


01 


04 


Identify contaminating variables 


01 


05 


State major hypotheses/philos position 


01 


06 


Observe relevant /related field opns 


01 


07 


Identify characteristics tgt population 


01 


08 


Determine problems to be addressed 


01 


09 


Determine sensitive areas in tgt context 


01 


10 


Determine priorities of problems 


01 


11 


Visualize application of technique/ideas 


01 


12 


Review of /familiarization w/materials 


01 


13 


Obtain literature relevant to problem 


01 


14 


Compare existing frameworks/procedures 


01 


15 


Operationally define terms/variables 


01 


16 


Identify/assess divergent points of view 



* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. It Is recorded In Columns NOQR on Form 07 for each 
statement of a task classifiable within Cluster NO-Ol. 
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Ql< (N(;-02) 


APPENDIX 4 (cont'd.) QR (NO-02) 

Primary Categories of Tasks (cont'd.) 


Set 

NO QR 
Coding* 


Primary Categories (Cluster NO-02) 


02 01 
02 02 
02 03 

02 04 

02 05 


Conceptualize the theoretical framework 
Conceptualize the product and context 
Formulate specif ic questions /hypotheses 
Determine nature of msmt objectives 
State problem in context of theory 


02 06 
02 07 


Create objectives 

Confer with colleagues/others re objects 



* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. It is recorded in Columns NOQR on Form 07 for each 
statement of a task classifiable within Cluster NO-02. 



€ 



QR (NO-03) 



APPENDIX 4 (cont'd.) 



Primary Categories of Tasks (cont'd.) 



QR (NO-03) 



Sec 




NO 


QR 




Coding* 


Primary Categories (Cluster NO-03) 


03 


01 


Identify decision situations of interest 


03 


02 


Isolate contaminating variables 


03 


03 


Determine relation of variables/compnts 


03 


04 


Select /specify population (s) involved 


03 


05 


Specify /select grouping /sampling 


03 


06 


Specify treatments/procedures/strategies 


03 


07 


Conceptualize output application 


03 


08 


Translate objectives/data into meaning 


03 


09 


Visualize/specif y format /appearance 


03 


10 


Specify/identify character of materialf. 


03 


12 


Estab tolerances/ef f ectiveness criteria 


03 


13 


Establish specs for output try-out/demo 


03 


15 


Specify site adaptations required 


03 


17 


Specify measures to be used 


03 


18 


Specify data/info processing/up-date 


03 


19 


Specify kind of data analysis to be used 


03 


20 


State overall design 


03 


22 


Specify data collection schedule 


03 


23 


Consider/design alternative approaches 


03 


24 


Visualize/speciiy effect of output 



* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. It is recorded in Columns NOQR on Form 07 for each 
statement of a task classifiable within Cluster NO-03. 



QR (NO-04) 



APPENDIX A (cont'd.) 

Primary Categories of Tasks (cont'd.) 



QR (NO-OA) 



Set 

NO QR 
Coding* 


Primary Categories (Cluster NO-OA) 


OA 


01 


Construct /assemble measurement tools 


OA 


02 


Construct/assemble components of output 


OA 


OA 


Secure/ select required materials/comps 


OA 


05 


Produce outpu t 


OA 


07 


Package output 


OA 


08 


Select/produce d^=ta processing/handling 


OA 


09 


Translate outcomes/content to user terms 


OA 


11 


Get permission to use copyright material 


OA 


12 


Invent new production method/instrument 


04 


13 


Construct/employ production tools/aids 


OA 


lA 


Adapt materials for local/specific use 


OA 


15 


Copy output for other media applications 


OA 


16 


Coach talent (filming , video , sound) 


OA 


17 


Organize presentation components 


OA 


18 


Provide materials to producers 


OA 


19 


Adapt content into processing format 



1 



* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. It is recorded in Columns NOQR on Form 07 for each 
statement of a task classifiable within Cluster NO-OA. 
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QR (NO-05) 



QR (NO-05) 



APPENDIX 4 (cont'd.) 



Primary Categories of Tasks (cont d.) 



Set 



NO 


QR 




Coding* 


Primary Categories (Cluster NO-05) 


05 


01 


Conduct/operate program to be measured 


05 


02 


Administer/ implement data instruments 


05 


03 


Record/score/classify/code raw data 


05 


04 


Transfer raw data into storage form 


05 


05 


Translate data into processing form 


05 


06 


Perform analyses as specified/ indicated 


05 


07 


Compile computational data 


C5 


08 


Prepare supporting documentation 


05 


09 


Interpret data 


05 


10 


Acquire art i f ac t s 


05 


11 


Try-out output/component for operability 


05 


12 


Specify job control lang for computer 


05 


13 


Identify sub-routines req by computer 


05 


14 


Specify parameters of sub-routines 


04 


15 


Sequence the compu ter program 


05 


16 


Determine constants inherent in analysis 


05 


17 


Specify format for computer printout 


05 


18 


Construct/maintain data displays 


05 


19 


Specify /identify data identity codes 



* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. It is recorded in Columns NOQR on Form 07 for each 
statement of a task classifiable within Cluster NO-05 . 
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QR (NO-06) 



APPENDIX 4 (cont'd.) 

Primary Categories of Tasks (cont'd.) 



QR (NO-06) 



Set 




NO 


QR 




Coding^ 


Primary Categories (Cluster NO-06) 


06 


01 


Check output against specifications 


06 


02 


Judge significance of performance result 


06 


03 


Determine achievement of objectives 


06 


05 


Identify unexpected/incongruent outcomes 


06 


06 


Judge output operability/ acceptability 


06 


07 


Judge output effectiveness /consistency 


06 


08 


Determine inadequate output components 


06 


10 


Confirm operability/validity of tools 


06 


11 


Judge effectiveness of data gathering 


06 


12 


Relate data to decision situations 


06 


14 


Note/make advisable modification (s) 


06 


15 


Proofread/edit written material 


06 


16 


Determine adeq of implementation process 


06 


17 


Check recommended changes specs 


06 


18 


Judge output against personal standards 


06 


19 


Edit video-audio- film materials 


06 


20 


Select best alternative method/procedure 



* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. It is recorded in Columns NOQR on Form 07 for each 
statement of a task classifiable within Cluster NO-06. 



QR (NO-07) 



APPENDIX 4 (cont'd.) 



QR (NO-07) 



Primary Categories of Tasks (cont'd.) 



Set 

NO QR 
Coding* 


Primary Categories (Cluster NO-07) 


07 


01 


Study/describe characteristics of output 


07 


08 


Negotiate for sale of copyrighted matter 


07 


09 


Prepare/write promotional materials 


07 


11 


Disseminate promotional materials 


07 


12 


Present / demonstrate output to user 


07 


13 


Negotiate for user try-out 


07 


14 


Negotiate user acceptance/use 


07 


15 


Provide for user training as required 


07 


17 


Distribute output/component 


07 


18 


Maintain distribution list 






* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. It is recorded in Columns NOQR on Form 07 for each 
statement of a task classifiable within Cluster NO-07. 
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QR (NO-21) APPENDIX 4 (cont'd.) 

Primary Categories of Tasks (cont'd.) 



Set 




NO 


QR 




Coding* 


Primary Categories (Cluster NO-21) 


21 


01 


Analyze work for competencies required 


21 


02 


Determine competencies available 


21 


03 


Formulate job desrriptions/qualif /salary 


21 


05 


Determine need for other points of view 


21 


06 


Maintain list of replacement personnel 


21 


07 


Make vacancies known 


21 


08 


Identify potential candidates 


21 


09 


Interview/screen candidates 


21 


10 


Select/hire/appoint personnel 


21 


11 


Provide training as appropriate 


21 


12 


Negotiate w/candidate to employ /partic 


21 


13 


Determine manpower needs 


21 


14 


Determine tasks to procure staff 


21 


15 


Recommend personnel for employment 


21 


16 


Obtain approval to hire 


21 


17 


Determine/assess best staffing pattern 


21 


18 


Obtain/assess candidate’s goals 


21 


19 


Review opns relative to existing staff 


21 


20 


Confer w/others re candidate selection 



QR (NO-21) 



* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. It is recorded in Columns NOQR on Form 07 for each 
statement of a task classifiable within Cluster NO-21. 




QR (NO-22) 



QR (NO-22) 



APPENDIX 4 (cont'd.) 



Primary Categories of Tasks (cont'd.) 



Set 

NO QR 
Coding’*f 


Primary Categories (Cluster NO-22) 


22 


01 


Specify accountability procedures 


22 


02 


Delegate responsibilities /authority 


22 


03 


Determine/verify/monitor budget balances 


22 


04 


Establish/maintain time lines 


22 


05 


Project cost estimates 


22 


06 


Determine nature/content of reporting 


22 


07 


Negotiate/renegotiate funding/objectives 


22 


08 


Approve disbursement of funds 


22 


09 


Monitor expenditure of time/svcs/money 


22 


10 


Report status as required/appropriate 


22 


11 


Explicate obj /outputs to be produced 


22 


12 


Explicate procedures /strategies /duties 


22 


13 


Interact with advisory services 


22 


14 


Issue production orders 


22 


15 


Prepare detailed spending plan 


22 


16 


Compare expenditure rate with plan 


22 


17 


Maintain duplicate records/files 


22 


18 


Insure optimal use of resources 


22 


19 


Maintain materials/use/fiscal records 


22 


20 


Develop /main tain inventory system 


22 


21 


Develop record keeping system 


22 


22 


Estimate/provide material/personnel 


22 


23 


Direct /request special reports 


22 


24 


Determine/comply w/sponsor*s procedures 


22 


25 


Pay bills/collect payments 


22 


26 


Divide work into appropriate segments 


22 


27 


Monitor/det ermine cost-effectiveness 


22 


28 


Specify/establish/adopt policies 


22 


29 


Oversee/monitor adherence to requirements 


22 


30 


Obtain staff input regarding time/funds 


22 


31 


Participate in production efforts as req 


22 


32 


Approve budget 



* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. It is recorded in Columns NOQR on Form 07 for each 
statement of a task classifiable within Cluster NO-22. 
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OR (NO-2'3) 


APPENDIX 4 (cont'ci.) qK (NO-23) 

Primary Categories of Tasks (cont'd.) 


Set 

NO QR 
Coding* 


Primary Categories (Cluster NO-23) 


23 01 

23 02 

23 03 

23 04 

23 05 


Determine tasks/services to be performed 
Determine facilities/f ield material req 
Identify/study possible sitss/services 
Selecc/obtain sites/facilities/services 
Negotiate contract /coop w/ sites /services 


23 06 

23 07 

23 08 

23 09 


Get permission to use copyright material 
Arrange visitation/schedule as needed 
Make public service/material/site needs 
Procure required too Is /equipment 



* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. It is recorded in Columns NOQR on Form 07 for each 
statement of a task classifiable within Cluster NO-23. 



QR (NO-24) 



APPENDIX 4 (cont’d.) 

Primary Categories of Tasks (cont'd.) 



QR (NO-24) 



Set 




NO 


QR 




Coding* 


Primary Categories (Cluster NO-24) 


24 


01 


Specify/recommend criteria for output 


24 


03 


Specify standards for tmg effectiveness 


24 


04 


Specify/develop improvement procedures 


24 


05 


Delegate quality control responsibility 


24 


06 


Monitor adherence to output specs 


24 


07 


Monitor personnel performance 


24 


08 


Monitor training effectiveness 


24 


09 


Initiate/direct quality control action 


24 


10 


Monitor progress/status of work 


24 


11 


Proof /critique products /performances 


24 


12 


Monitor adherence to policies 


24 


13 


Monitor decisions made 


24 


14 


Determine need for/present alternatives 


24 


15 


Identify quality control issues 


24 


16 


Negotiate quality issues with sponsor 


24 


17 


Direct/request critiques from others 


24 


18 


Monitor staff awareness of responsibility 


24 


19 


Approve potential staff for employment 


24 


20 


Monitor philosophical consistency of opn 


24 


21 


Adjust staff for operation compatibility 


24 


22 


Select outputs for further review 


24 


23 


Confirm accuracy of data inputs/outputs 


24 


24 


Monitor / j udge proj ect impact 


24 


25 


Elicit author approval re project changes 



me 



* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. It is recorded in Columns NOQR on Form 07 for each 
statement of a task classifiable within Cluster, NO-24 . 
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QR (NO-25) APPENDIX 4 (cont'd.) 

Primary Categories of Tasks (cont’d.) 



Set 




NO 


QR 




Coding* 


Primary Categories (Cluster NO-25) 


25 


01 


Specify/explain agency/project position 


25 


02 


Provide means for handling staff views 


25 


03 


Recognize merit 


25 


04 


Counsel with/be supportive of personnel 


25 


05 


Adjust tasks to fit interests/limits 


25 


06 


Recommend promotions/salary increases 


25 


07 


Encourage creativity 


25 


08 


Reconcile differences of opinion 


25 


09 


Encourage optimal self-direction 


25 


10 


Foster mutual respect 


25 


11 


Support staff in decisions they make 


25 


12 


Foster feelings of security in job 


25 


13 


Provide for social activities 


25 


14 


Act on behalf of staff in agency matters 


25 


15 


Nurture staff feelings of importance 



QR (NO 



* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. It is recorded in Columns NOQR on Form 07 for each 
statement of a task classifiable within Cluster NO-25. 
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QR (NO-26) 



QR 


(NO-26) 


APPENDIX 4 (cont'd.) 




- 


Primary Categories of Tasks (cont'd.) 


Set 




NO 


QR 




Coding* 


Primary Categories (Cluster NO-26) 


26 


01 


Provide for guidance within project 


26 


02 


Provide opportunities for growth 


26 


03 


Provide incentive for quality work 


26 


05 


Provide opportunities for exposure 


26 


06 


Specify ownership/authorship policies 


26 


07 


Encourage publication of important work 



* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. It is recorded in Columns NOQR on Form 07 for each 
statement of a task classifiable within Cluster NO-26. 
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QK (NO-27) 



QR (NO-27) 





Set 


NO 


QR 


Coding* 


27 


01 


27 


03 


27 


OA 


27 


06 



APPENDIX A (cont 'd. ) 



Primary Categories of Tasks (conr'd.) 



Primary Categories (Cluster NO-27) 

Provide optimal proximity of staff 
Provide for privacy 
Provide meeting /relaxation facility 
Optimize accessibility to equip/services 



* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. It is recorded in Columns NOQR on Form 07 for each 
statement of a task classifiable within Cluster NO-27. 



QR (NO-28) 



APPENDIX 4 (cont'd.) 

Primary Categories of Tasks (cont'd.) 



QR (NO-28) 



Set 



NO QR 
Coding* 


Primary Categories (Cluster NO-28) 


28 01 
28 02 
28 03 


Set demands in keeping with competencies 
Determine excessive/lov work loads 
Insure equity in application of rules 



* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. It is recorded in Columns NOQR on Form 07 for each 
statement of a task classifiable within Cluster NO-28. 



f’-; (NO-29) 



APPENDIX k (cont'd.) 

Primary Categories of Tasks (cont'd.) 



QR (NO-29) 



Set 




NO 


QR 




Coding* 


Primary Categories (Cluster NO-29) 


29 


01 


Monitor field setting views /concerns 


29 


02 


Give visibility to cooperating people 


29 


03 


Give visibility to cooperating agencies 


29 


OA 


Interact w/setting to elicit cooperation 


29 


05 


Communicate behavior required of setting 


29 


06 


Explain/prepare rationale for procedure to site 


29 


07 


Negotiate resolution of difference 


29 


08 


Locate/document sources of disagreement 


29 


09 


Adapt to accommodate differences 


29 


10 


Promote or maintain focus of effort 


29 


11 


Translate language into understood terms 


29 


12 


Provide for alternate output treatment 


29 


13 


Maintain contacts with field 


29 


lA 


Interact with setting to collaborate 


29 


15 


Interact w/setting to enhance 


29 


16 


Make materials/experiences available 


29 


17 


Provide foreign language translations 


29 


18 


Endorsement/support for field personnel 



* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. It is recorded in Columns NOQR on Form 07 for each 
statement of a task classifiable within Cluster NO-29. 
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QR (NO-30) 



APPENDIX 4 (cont'd.) 

Primary Categories of Tasks (cont'd.) 



QR (NO-30) 



Set 

NO QR 
Coding* 


Primary Categories (Cluster NO-30) 


30 


01 


Define project Information flow policies 


30 


02 


Determine/use information sources 


30 


03 


Insure that key decision points get info 


30 


04 


Provide communications equip/capability 


30 


05 


Define lines of communications 


30 


06 


Monitor information flow 


30 


07 


Explicate/employ ways of enhancing flow 


30 


08 


Coordinate people interactions 




* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. It Is recorded In Columns NOQR on Form 07 for each 
statement of a task classifiable within Cluster NO-30. 
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QR (NO-31) 



Set 



APPENDIX 4 (cont’d.) 

Primary Categories of Tasks (cont’d.) 



QR (NO- 31) 



NO 


QR 




Coding* 


Primary Categories (Cluster NO-31) 


31 


01 


Plan /conduct staff meetings /conferences 


31 


02 


Make formal presentations 


31 


03 


Produce/circulate bullet ins /memos /news 


31 


04 


Lead/participat^i in^ jtask-oriented groups 


31 


05 


Join in^^le’ad info meetings/contacts 


31 


06 


s* 

Receive and react to relevant input 


31 


07 


Communicate decisions made 


31 


08 


Communicate production guidelines 


31 


09 


Keep involved personnel informed generally 


31 


10 


Communicate relevant experience info 


31 


11 


Advise on admin concerns w/in expertise 


31 


12 


Refer problem to appropriate specialists 


31 


13 


Record events (e.g., minutes, processes) 


31 


14 


Seirve in consulting capacity to project 


31 


15 


Advise technically within expertise 



* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. ^It^is recorded in Columns NOQR on Form 07 for each 
statement of a task classifiable within Cluster NO-31. 
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QR (NO-32) APPENDIX 4 (cont'd.) 

Primary Categories of Tasks (cont'd.) 



Set 




NO 


QR 




Coding* 


Primary Categories (Cluster NO-32) 


32 


01 


Write professional papers 


32 


02 


Frepare/dissemlnate promotional info 


32 


03 


Act as consultant 


32 


04 


Report proj ect at agency/other iriee tings 


32 


06 


Interact with dissemination ser^^ices 


32 


07 


Provide info to dissemination services 


32 


08 


Interact with others regarding promotion 


32 


09 


Enlist community help in dissemination 


32 


10 


Insure appropiTlate visibility of project 


32 


11 


Approve articles/papers for publication 


32 


12 


Prepare project descriptive materials 




* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. It Is recorded In Columns NOQR on Form 07 for each 
statement of a task classifiable within Cluster NO-32. 
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APPENDIX 4 (conc’d.) 



QR (NO-33) 



Primary Categories of Tasks (cont'd.) 



Set 

NO QR 
Coding* 


Primary Categories (Cluster NO-3 3) 


33 


01 


Specify/determine key decision points 


33 


02 


Specify kinds of decisions at each level 


33 


03 


Make decisions 


33 


04 


Specify who needs what kinds of info 


33 


06 


Specify vehicles for decision-making 


33 


07 


Participate in decision-making process 


33 


08 


Identify /specify sources of problems 


33 


09 


Request support for decisions made 


33 


10 


Establish organizational structure 


33 


11 


Get partic in decisions/conceptualizing 



* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. It is recorded in Columns NOQR on Form 07 for each 
statement of a task classifiable within Cluster NO-33. 
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APPENDIX 4 (cont'd.) 
Category Set S 



S 



Structure of Enablers 



Set S 

Coding* Structure 

1 Knowledge 

2 Skill 



3 



Sensitivity 



* Responsibility for assignment of this code rests with the Interviewer 
as each Interview Is recapped on Fora 08. As each statement of an enabler 
Is recorded, the appropriate code Is entered In Column S. The Interviewer 
records the code that best reflects the Interview context In which the 
statement occurred and the Intent of the statement. This code Is subject 
to review and confirmation by the Coding Resolution Team. 
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Set T 
CoditiR* 

1 

2 

3 



APPENDIX 4 (cont'd.) 
Category Set T 

Self /Others Report (for Enablers) 

Self/Others Report (for Enablers) 

Self 

Others 

Team 




* Responsibility for assignment of this code rests with the interviewer 
as each interview is recapped on Form 08. As each statement of an 
enabler is recorded, the appropriate Self /Others code is entered in 
Column T. 0 C* / ■ 

kJ' V 
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APPENDIX 4 (cont'd.) 
Category Set UV 



UV (S-1) 



Primary Categories of Enablers 






Primary Categories of Knowledge Enablers (Category Set S-1) 

Primary Categories (S-1) 



Set UV 
Coding* 



01 

02 

03 

04 

05 

06 

07 

08 

09 

10 

11 

12 

13 

14 
16 

17 

18 

19 

20 
21 

22 

23 

24 

25 

26 

27 

28 

29 

30 



Standard school subjects 
Subjects learned In courses 
Subjects related to RDD&E 
Technlcal/professlonal topics 
Project focus topics: external 

Project variables: external 

Project operation: general 

Project operation: specific 

Scheduling & organizing 
Staff status/responslbllltles 

Fiscal matters 
Resources: personnel 

Resources: money 

Resources: time 

Guidelines for reporting 

Writing styles 

Staff competencies/interests 
Technical terminology/language 
Sponsor concerns 
Management techniques 

Use of equipment/systems 
Char's of target audience 
Process implementation (pro j ) 
Sources of Info/materlals 
Incorrect data entries/posting 

Potential field settings 
Group dyn/declslon processes 
Need for material /Info 
Errors in strategies/ judgment 



* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. It is recorded in Columns UV on Form 08 for each state- 
ment of an enabling knowledge associated with the output being analyzed. 
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UV (S-2) APPENDIX 4 (cont'd.) 

Primary Categories of Enablers (cont'd.) 



Primary Categorie 

Set UV 
Coding* 

01 

02 

03 

04 

05 

06 

07 

08 

09 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 

34 

35 

36 

37 

38 

39 

40 



ERIC 



of Skill Enablers (Category Set S~2) 

Primary Categories (S-2) 

Teaching 

Facilitating people interactions 
Translating content to media 
Using/ applying feedback 
Programming project events 

Programming subject matter 
Programming technical equip 
Analytical reading/study 
Analytical problem solving 
Analytical data handling 

Disciplining self 
Disciplining others 
Listening 
Writing 

Presenting orally 

Using media 
Interpreting language 
Finding fits/integrating 
Planning/conceptualizing 
Exercising judgment 

Tracking activities/goals 
Estimating expenses/resources 
Persuading/justifying 
Explicating goals/procedures 
Applying measurement tools 

Locating/maintaining info 
Using equipment/systems 
Running task oriented meetings 
Getting others to perform 
Adapting to situation/demands 

Taking another's perspective 
Identifying/correcting errors 
Graphically illustrating 
Coordinating activities 
Communicating clearly 

Cataloging/classifying 
Assessing ski 11s /potentials 
Using resources effectively 
Constructiilg measurement tools 
Defining terms/problems 
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APPENDIX 4 (cont'd.) 



UV (S-2) 





Primary Categories of Enablers (cont'd.) 


Set UV 
Coding* 


Primary Categories (S-2, cont'd.) 


41 

42 

43 

44 

45 


Maintaining physical condition 
Eliciting responses 
Instilling confidence 
Fiscal accounting 
Assessing personal performance 


46 

47 

48 

49 


Placing others at ease 
Recalling 

Establishing credibility 
Interacting productively 



* Responsibility for the assignment of this code rests with the Coding 
Resolution Team. It is recorded In Columns UV on Form 08 for each state 
ment of an enabling skill associated with the output being analyzed. 



lA/ rs-'3) APPENDIX A (cont*d.) 

Primary Categories of Enablers (cont*d.) 
Primary Categories of Sensitivity Enablers (Category Set S 



Set UV 

Cod in g^^ Primary Categories (S-3) 



01 

02 

03 

04 

05 



Values of self and others 
Capabilities and limitations 
Needs of self and others 
Interactions of self & others 
Context of subject matter 



06 Worth in disciplines /methods 

07 Context of objectives 

08 Worth in objectives 

09 Awareness of alternatives 

10 Awareness of structure 



11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 



Awareness of method 

Role of catalyst /synthesizer 

Language barriers 

Reality in goal setting 

Degrees of freedom to deviate 

Existing value systems 
Personality of others 
Potential conflict of interest 
Supportiveness required 
Unstated obligations 

Limitations of analyses/data 
Responses of target audiences 
Cos t /bene fit factors 
Sources of error 
Individual differences 



26 Recognition of data needs 

27 Acceptability of output 

28 Admitting error/ adapting 

29 **Willingness to experiment 

30 Response sets of target audiences 



31 Nature/scope of output 

32 Reality in spending 

33 Need for excellence in work 

34 **Willingness to accept guidance 

35 Need to communicate effectively 



36 

37 

38 

39 

40 



**Feeling of personal security 
**Willingness to delegate 
Emerging directions 
Limits of **one^shot" efforts 
Contractual/stated obligations 



UV (S-3) 
-3) 



UV (S-3) 



UV (S-3) 



APPENDIX 4 (cont'd.) 

Primary Categories of Enablers (cont'd.) 



1 Set UV 

Coding* 


Primary Categories (S-3 cont'd.) 


^ 42 

43 

44 

45 


**Inquisitiveness 

**Enthusiasm 

**Challenged by activities 
**Interested in activities 
Sound variability 


46 

47 

48 

49 

50 


**Sense of humor 
**Willingness to work as needed 
**Common sense 
**Willingness to learn 
** Intellectual openness 


r 51 

i 52 

53 

54 

55 


Possible points of confusion 
Awareness of staff affect 
**Creative 

Manageability of data 
**Respect for/ trust in others 


56 

57 

58 


**Habit of thoroughness 
Deadlines 

**Willingness to support staff 



* Responsibility 


for the assignment of this code rests with the Coding 



Resolution Team. It is recorded in Columns UV on Form 08 for each 
statement of an enabling sensitivity associated with the output being 
analyzed. 

** Items emerging from data collection, classified as sensitivities, 
but which appear to be habits or personality characteristics. 
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APPENDIX 4 (cont'd.) 



W 



W 

Set W 
Coding* 

1 

2 

3 

A 

5 

6 



Category Set W 
Output Completion Stage 

Completion Stage 

Completed a long time ago (over one year ago) 
Completed some time ago (three to twelve months ago) 
Recently completed (within last three months) 
Currently In process 
Not yet underway 

On-golng (generally continuous throughout project) 



* Responsibility for assignment of this code rests with the Interview 
team leader. Determination of completion stage for every output listed 
for a project Is made on site and the appropriate code entered In 
Column W on Form 05B, 





X 



APPENDIX 4 (cont'd.) 



X 





Category Set X 


Set X 

' ^ Coding* 

1 


Project Role of Inteirviewee 
P’'oject Roles 

Principal Investigator (contractual responsibility) 


2 


Project Director (if other than P.I.) 


3 


Top or intermediate-level project manager or coor- 
dinator 


4 

► 

5 


First-line supervisor (supervising three or more) 
Member, professional staff 


\ 

6 


Member, clerical support staff 


7 


Member, technical support staff 


1 8 


Advisor or consultant 


9 


Other 



* Responsibility for the assignment of this code rests with the inter- 
viewer. Determination of role is made as a function of interview, con- 
firmed with interviewee and the appropriate code entered as Item 47 in 
Questionnaire Form 03. 
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Y 

Set Y 
Coding* 

1 

2 

3 

4 



APPENDIX 4 (cont’d.) 
Category Set Y 
Project Focus 

Project Focus 
Research (R) 

Development (D) 

Diffusion (D) 

Evaluation (E) 



Y 



* This code to be tentatively given each project prior to site visita- 
tion by interview team as a function of site selection. Responsibility 
for confirming and modifying the code rests with the team leader and the 
Coding Resolution Team. The code is entered in Column Y on Form 05B. 
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APPENDIX 4 (cont'd.) 
Category Set Z 



Character of Supported Focal Output 

Set Z 

Coding* Character of Supported Focal Output 

1 Knowledge 

2 Technology 

3 Implementation 

4 Information 



* This code is entered in Column Z on Form 05B. It corresponds to the 
Category Set C code given the focal output to which it is linked on the 
output map. Responsibility for this entry rests with the Coding Reso- 
lution Team with confirmation by the site visitation team leader. 



O 
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APPENDIX 5 



Decision Rules for Coding Output and 
Work Requirement Data 

Category Set A; Structure of Output 



Purpose of the Classification 

The classification of an o.tput according to the structure it repre- 
sents permits discrimination of outputs having varying degrees of tan- 
gibility. (It is recorded in Column A of Form 05B for each output iden- 
tified in project and in Cell A of Forms 06, 07, and 08 for each output 

being analyzed.) 



Definitions 



Code Structure 



1 Product - Something tangible or "hard” produced as an outcome 
of targeted work effort, c' .crete in form, and transportable 
at a given" point in time. 

Decision ru]es: (a) the site or informant provides 

a copy of the product; or 
(b) the site or informant specifies 
that the output is in physical 
form or relates to it as such. 

Examples: Questionnaire; time line chart ; policy manual . 

2 Event - An observable transaction or set of behaviors produced 
as an outcome of targeted work effort. 

Decision rules: (a) the site or informant specifies 

an occurrence significant to the 
project toward which there was 
focused work effort; 

(b) the output descriptor clearly 

states or implies a set or class 
of behaviors occurring within 
time and content constraints. 

Examples: Field test; conference; workshop; speech . 

3 Condition - A desired circumstance expected to endure within 
the life of a project, or beyond as a result of the project, 
produced as an outcome of targeted work 'iffort. 

Decision rules: (a) the site cr informant specifies 

an output in phenomenological 
terms, i.e., in terms descriptive 
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APPENDIX 5 (cont’d,) 



of a circuniwS tantial characteris- 
tic ; 

(b) the output descriptor refers 
directly to a characteristic 
rather than products or events 
which may lead to it. 

Examples: Parent involvement; staff welfare; supportive 

atmosphere. 



Discussion of Category Set A Decision Rules 

Criteria for output identification (see Chapter I) interact directly 
with criteria for classification within this category set, and the terms 
used to label the ou^"puts identified is critical. For example, "coor- 
dinating project activities" suggests the site views a significant out- 
put as a process or set of behaviors, hence it is classified as an event , 
On the other hand, "coordinated project activities" suggests a condition 
being sought after by a site. Critical to the labeling and classifica- 
tion of this output then, is a sensitivity to the way in which the site 
or informant views the end (output) being sought. 



Category Set b; Levels of Output 



Purpose of the Classification 

The classification of an output according to the level at which it 
appears peirmits distinguishing those that represent ends from those that 
represent components and those that are facilitative, (It is recorded in 
Column B on Form 05B for each output identifiea in a project and in Cell B 
of Foirms 06, 07, and 08 for each output being analyzed,) 4 ' 



Definitions 



Code Level 

1 Focal - An outcome of the targeted work, effort of a project 

that is regarded as central to the contractual obligation, 

(See Glossary) 

Decision rules : (a) the output is clearly identified 

and specified as a contractual 
obligation; and 

(b) the output is clearly identified 
and specified as being an end 
outcome of work effort which may 
optimize the consequences of the 
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Code 



2 



3 



project as a whole. 

Examples: of (a): Catalog of objectives; institutes; 

parent involvement ; final report ; 
of (b): Newsletter (for outside distribution); 

speeches (for dissemination of project as 
opposed to, e.g., fund generating activity). 

Level 

Component - An outcome of targeted work effort that constitutes 
an identifiable part of or integral ingredient in a focal output. 



Decision rules: 


(a) 


the output is not a focal output 
but is identified as integral 






part of it; or 




(b) 


the output is a prototype of the 
focal output which does not ful- 
fill, at that stage, all contrac- 
tual requirements of the focal 
output . 



Examples: Tryout version of catalog of objectives; meas- 

ures of objectives achievement (when part of unit). 



Facilitator - An outcome of targeted work effort that makes 
easier or assists the accomplishment of other outputs of a 
project and which is not an instance of a focal or component 
output . 

Decision rules: (a) the output clearly is not a focal 

effort of the project, nor can 
its identity be directly perceived 
within it; 

(b) the output is related to the pro- 
ject in that it makes easier or 
assists in the performance of a 
sound , cohes ive proj ect operation ; 
or 

(c) the output is related to the pro- 
ject in that it makes easier or 
assists in fulfilling various 
terms of a contract. 

Examples: Any data collection instrument except where 

instrumentation in that form is explicitly 
stated as a project focus; timeline chart; staff 
meeting; atmosphere for constructive criticism. 



Discussion of Category Set B Decision Rules 

In applying Category Set B classifications to an output, the level 
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of an output must be viewed in relation to project ends (the focal out- 
puts) ; this is as opposed to a **nested-set" view, e.g,, staff welfare as 
a component of staff morale. Having established focal outputs, component 
outputs are those which consist of significant dimensions or **pieces” of 
those project ends. Facilitating outputs become, by exclusion, all other 
outputs relevant to project efforts or concerns. Technical tools of a 
project, unless specifically stated as outputs to be delivered, are thus 
considered to be facilitators even though they may be contained in a final 
report. A final report which merely satisfies accountability require- 
ments of a project, while focal in nature, does not qualify the tools as 
components in a project sense. On the other hand, if a technical report 
is identified as a contractual obligation, the tools may well be of a 
component nature in a project sense. In any event, the purpose is to 
discriminate those outputs for which direct accountability exists from 
those which support that accountability. 



Category Set C: Output Character 



Purpose of the Classification 

The classification of an output according to its character is to 
permit one to distinguish outputs on the basis of whether they represent 
knowledge , technology , implementation . or information outcomes. (It is 
recorded in Column C of Form 05B for each output identified in a project 
and in Cell C of Forms 06, 07, and 08 for each output being analyzed.) 



Definitions 



Code Character 



1 Knowledge - An outcome of work effort (output) which is viewed 

as a newly inferred and generalizable idea or principle taken 
from facts or data, which adds to that which is known and pro- 
vides a new base from which to view other phenomena. 

Decision rules: (a) the output has the characteris- 

tic of having been empirically 
tested by the project through 
the process generating it; and 
(b) the output has the characteris-- 
tic of an idea, principle, etc., 
derived from data, or is a 
report of those things having 
utility or application beyond 
the context in which it was 
generated. 

Examples: Nature of reading; conceptual papers (empir- 
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ically derived and tested); experimental 
(research) data; data analysis and interpre- 
tation (research data) . 



Code Character 

2 Technology - An outcome of work effort which is viewed as a 
reliable strategy, procedure, hardware, or set of materials 
designed to bring about a particular outcome or to perform 
a defined operation. 

Decisioh tules : (a) the output has the essential 

characteristic of providing a 
framework for accomplishing a 
specified end; and , 

(b) the output has the essential 
characteristic reached as a 
result of its application. 

Examples: A questionnaire; single concept film; field 

test plan. 

3 Implementation - An outcome of work effort which is viewed as 
an instance of the adoption and/or utilization of knowledge, 
technology, and/or information. 

Decision rules : (a) the output has the characteris- 

tic of transmitting knowledge , 
information and/or a technology; 
and , 

(b) the output has the characteris- 
tic of others adopting and/or 
util izing knowledge , informa- 
tion, or technology within cer- 
tain specified parameters . 

Examples: Commercial edition of a catalog of objectives; 

popular magazine publication supporting a pro- 
gram, final report (when serving accountabil- 
ity and not meeting other criteria). 

4 Information - An outcome of work effort which is viewed as 
trustworthy facts or data descriptive of something without 
reference to interpretive or value judgments relating to that 
description. 

Decision rules: (a) the output has the characteris- 

tic of having been legitimized 
through the processes generating 
it; and , 

(b) the output has the characteris- 
tic of data, or a report of that 

O 
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data, having utility or applica- 
tion within the context of its 
generation. 

Examples: Description of resource management system; 

s tat is tical and reference reports ; progress 
reports; performance testing data; criterion 
referenced test item pool; normative data; 
coordinated field- tes t site data collection • 



Discussion of Category Set C Decision Rules 

The classification of an output in relation to its character must 
be based on what the output The effort is to determine whether the 

output, on the bas is of what it is (be it product , event , or condi tion) , 
satisfies requirements (a) and (b) in any single character, i.e., knowl- 
edge, technology, implementation or information. If it does, that clas- 
sif ication is given the output . Care mus t be taken to insure that an 
output is not classified by virtue of the perceived processes which went 
into its generation, but rather whether it is, in fact, represented as 
an output having the specified characteristics. Likewise, it is not 
appropriate to classify an output in terms of a sense of its purpose. 
Thus, if multiple classifications are seen as possible, a second exam- 
ination of the output must be made to determine the single classifica- 
tion which best describes what it represents. 



Category Set **a'*: Output Function 



Purpose of the Classification 

The classification of an output according to its function, i.e., 
policy, management or product ion per mi ts examination of the degree to 
which those who accomplish those outputs correlate with those whose 
roles are also so designated. (It is recorded in Column a of Form 05B 
for each output identified in a project.) 



Definitions 



Output Function - That classification given an output in accordance 
with the policy setting, management, or production service it 
renders to the course of project work. (See related definitions 
for policy, management, and production functions.) 

Code Function 

1 P olicy - A classification given an outcom<^ of work effort 

which establishes the principles or guidelines to be attended 
b; a project as a whole. 
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Decision rules : 



the output has the essential 



characteristic of influence on 
the project as a whole rather 
than on any specific set of out- 
puts; and , 

(b) the output has the characteristic 
of setting forth the basic para- 
meters of the project , the guide- 
lines and principles to be 
attended by overall operations 
and the resources which may be 
brought to bear in achieving the 
desired end. 

(c) an arbitrary decision rule 
excludes project proposals from 
this classification . (See defi- 
nition, decision rules, and exam- 
ples of management oriented out- 
puts in which the typical ele- 
ments of a proposal appear as man- 
agement in function.) 



for quality control; seminars on goals to 
be pursued; organizational management 
structures; technical paper for new person- 
nel. 



acteristic of serving information 
needs with respect to outputs or 
operations of the project; and/or 

(b) the output specifies timelines, 
budget line items, and/or person- 
nel assignments relative to pro- 
ject operations; and/or 

(c) the output has the characteristic 
of an operational decision affect- 
ing project outputs and/or opera- 
tions, other than those having 
policy-oriented functions; and/or 

(d) the output has the essential char- 
acteristic of serving accountabil- 
ity obligations; and/or 



Examples: Operational plans and policies; criteria 



Code 



Function 



2 



Management - A classification given an outcome of work which 
orchestrates resources (time, personnel, materials, space, 
information) available to a project in the realization of out- 
comes expected from it, or is a report of that orchestration. 
(See discussion section for this category set.) 



Decision rules: (a) the output has the essential char- 
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(e) the output .specifies what l.s to 
be produced. 

Examples: Final report (accountability); progress/finan- 

cial reports ; questionnaires ; observation sys- 
tem; specification of materials; PERT chart; 
list of sites; quality control schema; journal 
article; field test data; directory; staff 
rneeting/hiring; acquisition of materials /funds; 
product quality . 

Code Function 

3 Production - A classification given an outcome of work effort 

which has the characteristic of building directly on the fab- 
rication efforts of a project. 

the output has the essential char- 
acteristic of comprising elements 
of, or are inherent in, the focal 
outputs being sought, as opposed 
to those which express or coerce 
various qualities of them; ^r 
the output has the characteristic 
of specifying how an output meet- 
ing the requirements of (a) is to 
be achieved (in terms of specific , 
discrete behaviors as opposed to 
sets of behaviors), 
focal outputs having contractual 
characteristics, along with com- 
ponent outputs, receive this clas- 
sification , as do facilitating 
outputs meeting the requirements 
of (b). 

Examples : Single concept film; parent involvement (where 

such is proclaimed to be a focal or component 
element contractually defined) ; research report ; 
conference (where that is a contracted-f or ele- 
ment of the effort) ; test data (where it is 
contracted for as in evaluation or research 
efforts); work conference; coordinated curric- 
ulum. 



Decision rules : (a) 



(b) 



(c) 



Discussion of Category Set '*a** Decision Rules 



The classification of an outpu* in relation to its function must be 
based on what an output does ia relation to the overall goals of a 
specified effort irrespective of the roles of those accomplishing them . 
Central to this classification is the assumption that outputs, in and of 
themselves, serve either a policy, management, or production function 
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relative to the objectives of aii effort. In this respect, many outputs 
(e.g., field test instruments, observation systems, etc.) have tradition- 
ally been thought of as elements of production efforts, and the tendency 
may be to classify them as having production functions. However, where 
the function of such outputs is, for example, clearly to provide for 
quality characteristics of another output, and nothing more, the classi- 
fication must be ’’management." Identical outputs may have, then, differ- 
ent functions in different projects or efforts depending upon the ends 
being sought. Care must be taken to examine critically the contribution 
of each output independently from other classifications and traditional 
ways of thinking about production activity in the larger sense. Success- 
fully classifying the functions of outputs in this manner permits, in 
part, such analyses as how the standards, tasks, or enablers (see defi- 
nitions) may vary for identical outputs when they serve different func- 
ti ons , and how management output respons ibili ties are dis tributed among 
management and other working personnel. 



Category Set DE; Output Cluster Categories 



Purpose of the Classification 

The classification of outputs to clusters facilitates retrieval 
of large amounts of information relative to those outputs having general 
qualities in common. (It is recorded in Columns DE of Form 05B for each 
output identified in a project and in Cells DE of Forms 06, 07, and 08 
for each output being analyzed.) 



Definitions 



Output Cluster Category - A classification given an outcome of work 
effort which identifies it as one of a number of different outputs 
having certain general qualities in common. 

Decision Rules: (a) the output has the characteristic 

usually associated with one of 
the output types listed in Cate- 
gory Set DE (see Appendix 4) ; and 

(b) the output has the characteristic 
of essentially serving the opera- 
tion implied by one of the output 
categories listed therein; 

(c) the output is the basis for estab- 
lishing a new output cluster cate- 
gory not :herwise represented in 
the DE set, ana a new clas.sifica- 
tion number is established for 
the output. 
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(d) the decision to add cluster cate- 
gories is dependent upon the level 
of specificity desired in analyz- 
ing clusters of outputs , keeping 
in mind the Category Set FGHI rep- 
resents categories of outputs at 
a specific level. 

Examples; Reports /contracts would include "final report," 
"personal services contract," "interim report," 
"progress report," etc. (See DE and FGHI Cate- 
gories in Appendix 4 for complete listing.) 



Discussion of Category Set DE Decision Rules 

In the development of Category Se^ DE , consideration is given to 
outputs as they appear to have operational similarities . Further, the 
set is to facilitate retrieval of data on large numbers of outputs having 
similar gross characteristics which are of interest to trainers. Thus 
the set serves not only as another means of classifying an output, but 
also as an index for easy access to meaningful clusters of output data. 

The first decision to be made in classifying an output as to its cluster 
category involves determining whether, in the context in which it was 
created, it has an operational relationship to any of the clusters listed. 
It will be noted that there are mutually exclusive clusters for each 
output structure (i.e., product, event, and condition). If an opera- 
tional relationship exists and is easily identified, that classification 
is given. If uncertain choices exist between two or more clusters, care- 
Tul examination of categories listed within the set may clarify the 
classification. Should it be decided a new cluster category is repre- 
sented, care should be taken to establish a cluster label at a gross 
enough level to permit other different outputs having like operational 
characteristics to be classified within it. The label must, however, be 
specific enough to preclude its confusion with other clusters already 
established. 



Category Set FGHI; Primary Output Categories 



Purj)ose of the Classification 

The '^.lassificaticn of outputs according to a primary category, 
allows for retrieval of all information relative to specific aggregated 
outputs, each of which r fleets relatively identical features and ele- 
ments. (It is recorded in Columns FGHI of Form 05B for each output iden- 
tified in a project and in Cells FGHI of Forms 06, 07, and 08 for each 
output being anclyced.) 
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Definitions 



Primary Output Category - A basic classification given an output 
in which the category assigned reflects the fundamental and essen- 
tial elements which mark the output as unique. Thus it is, for all 
practical purposes, a statement. 



Decision rules : 



(a) the output has all the identifi- 
able characteristics usually 
associated with one of the cate- 
gories listed in Category Set 
FGHI; and 

(b) the category being considered for 
classifying the output contains 
the same set of qualifiers in 

its description as does the out- 
put; or 

(c) the output is considered to have 
characterisitics or qualifiers 
which y as a set, makes the out- 
put different from any of those 
in the category list, and a new 
primary category is created. 

(d) the decision to add an output to 
the primary category list is 
dependent on whether, for all 
practical purposes, there is an 
exact match of characteristics 
and qualifiers between the items 
being classified and any of those 
already contained in the Category 
Set. Difference in level of des- 
criptive detail in the outpu^: and 
Category label sufficient to 
warrant a new item. 



Examples: Questionnaire is different from attitude ques- 

tionnaire; single concept film is different 
from film strip, slide film; a slide film on 
parts of speech is different from a s^ide film 
on the alphabet; a progress report is differ- 
ent from an interim report , etc. 



Discussion of Category Set FGHI Decision Rules 



In the d ' .elopment of Category Set FGHI, consideration is given to 
outputs as they appear to have operational differences . Further, this 
set is to represent a listing of the specific anr. d-^screte outputs as 
identified by project personnel , irrespective of leveJ. of explicitness, 
in order that the data collected for those analyzed maintain specific 
output integrity. Care must be taken in classifying an output to an 
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existing category, or in deciding to add a new category, to examine the 
characteristics of ^he output per se, rather than the tasks involved in 
accomplishing it. The reason for this is that the manner in which dif- 
ferent people see the tasks necessary to accomplish an identical output 
may vary resulting in the tasks being stated differently. Following 
this principle permits the generation of an increasingly complete range 
of data relative to specific outputs. In addition, should one choose 
to utilize alternative clusterings different from the DE Category Set, 
the outputs listed are discrete at the level described, and may be 
reclustered at face value . 



Category Set J: Structure Categories of Standards 



Purpose of the Classification 

The classification of a standard by the structure category it repre- 
sents permits discriminating those standards by which people judge the 
adequacy of an output from those standards by which people judge the 
adequacy of processes and/or operations involved with the accomplishment 
of that output. (It is recorded in Column J of Form 06 for each state- 
ment of a standard associated with the output being analyzed.) 



Definitions 



Structure of a standard - The characteristics of a statement of a 
standard which identify it as a measure of the adequacy of an out- 
put or as measure of the adequacy of processes associated with the 
accomplishment of that output. 

Code Structure Category 

1 Output Standard - A specific criterion applied to, or level of 

excellence expected of, a particular output, i.e., a criterion 
by which the adequacy of an output is judged. 

Decision rules: (a) the standard has the character- 

istic of referring to an output 
in terms of a level of perfor- 
m£mce expected of ojc 
(b) the standard has the characteris- 
tic of referring to an ouput in 
terms of the nature of its compo- 
sition . 

Examples : ‘‘Consistency in output performance . “ 

“Successfully cons train ts /guides an implemen- 
tation/production effort. “ 

“Compliance of output with sponsor guidelines.” 

O 
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'’Terminolopj appropriate for target (must 
refer to terms).” 

Code Structure Category 

2 Process / Operations Standard -- A criterion applied to, or level 

of excellence expected of, the processes and/or operations 
conducted in the achievement of an output, i.e., a criterion 
by which the adequacy of the processes and/or operations 
relating to achieving an output are judged. 

Decision rules : (a) the standard has the characterise 

tic of referring to uhe processes 
and/or operations involved with 
achieving an output in terms of 
a level of performance expected 
from them; or 

(b) the standard has the character- 
istic of referring to an output 
performance as a measure of the 
adequacy of the processes or 
operations which went into its 
achievement . (Discussion of 
Category Set J has special rel- 
evance to this criterion .) 

Examples: "Adequate operations conducted within budget.” 

"Manner of personnel performance fosters 
respect 

"Minimum redoing/debugging/correcting required.” 
"Follow-on proposals are funded.” 

"Feedback occurs.” 



Discussion of Category Set J Decision Rules 

The ability to discriminate between those standards which relate to 
an output ard those which relate to the processes and/or operations con- 
ducted in creating that output is occasionally difficult unless the 
object of the standard is specified. (The recap statements are to 
include sufficient information to indicate to which structure category 
the standard has reference.) Where similar standards are used to Judge 
the output as well as the processes, the primary categories (Category 
Set LM) for each are worded differently in order to facilitate proper 
identification of the structure category represented. To that extent, 
the listings of standards v^ithin each of the related LM sets may be 
used as additional examples for making this discrimination. 



APPENDIX 5 (cont’d.) 



Category Set K: Self/Qthers Report for Standards 



Purpose of the Classification 

The self/others classification for standards permits discrimi- 
nating those standards which are perceived to be imposed on an output 
by those other than the individual providing the increments of data 
being sought about an output. (It is recorded in Column K of Form 06 
for each statement of a standard associated with the output being 
analyzed. ) 



Definitions 



Code Self/Qthers Report for Standards 

1 Self (reported by interviewee as a standard he holds for 

the output being judged and/or the work performed in its pro- 
duction). 

2 Others (reported by interviewee as a standard held by others 
which is relevant to the output being judged and/or the work 
performed in its production) . 

3 Team (reported by interviewee as a Standard held by the inter- 
viewee and others as relevant to the output being judged and/ 
or the combined efforts in its production). 



Discussion of Category Set K 

The coding represented by this category set must be made by the 
interviewer or individual recapping audio- tapes of interview on the 
basis of the interview protocol. Instructions to interviewers specify 
that it must be clear whether the standard reported by the interviewee 
is held essentially by himself, others, or himself ^nd others as a team. 



Category Set LM; Standards 



Purpose of »^l.e Classification 

The purpose of this category set is to quantify the various stand- 
ards expressed for outputs, including the various ways or levels of 
Specificity with which they are expressed, and the frequencies with 
which they are cited. Such quantification of standards huld for each 
output facilitates analys.^s of the manner in which various standards 
may distribute across outputs of differing classes, levels, foci, and 
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I 



functions. (The classif.^catioT^ is recorded in Columns LM of Form 06 
for each statement of a s^:;ndard associf^t^H with the output being ana- 
l3'eed. ) 



Definitions 



Standard - A specific criterion applied to, level of excellence 
expected of, or a criterion by which judgments of adequacy are 
made about an output or the processes associated with its gener- 
ation. 



Dec i . ; on rules : 



(a) the standard, as expressed, must 
correspond for all intents ai d 
purposes with the nxianing and 
level of detail of one of the 
items in Category Set LM; or 

(b) the standard, as expressed, 
does not correspond in meaning 
and level of detail with any 
of the it«^ms and is added to 
the appropriate structure cate- 
gory (Set J) list. 



Examples: Primary category "Terminology Appropriate" 

includes: "the words are right for the tar- 
get "the terms used convey the right 

meaning Primary category "Feedback 

Occurs" includes: "people respond (to ques- 

tions, etc.)"; "people provide constructive 
criticism. " 



Discussion of Category Set LM Decision Rules 

The level of detail being sought for a statement of a standard may 
well be greater than that which infoimants can provide readily. Care 
must be taken to examine a statement carefully with respect to its 
intended meaning before choosing to select or add a category. Where 
doubt exists, it is usually profitable to discuss the ineaning with the 
informant (where coding is taking place on site) or the interviewer who 
collected the data. Also, it is essential, to the extent possible to 
avoid inferences about the meaning. Occasionally statements will 
app^.ar that contain more than one of the elements listed in the cate- 
gory set. In\ this case, it is permissible to paraphrase the statement 
into separate items and code each independently, since it is the ele- 
ments that are important. 
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C ategory Set NO: Cluster Categories of Ta^ks 



Purpose of the Classification 

The classification of tasks to cluster categories attends to two 
factors involved with the handling of information coming from a context 
having infinite variations. Of primary consideration in making this 
classification is the clustering of vide ranging task statements accord- 
ing to general process characteristics which they appear to have in 
commor . Secondarily, the labels given such qualities provides an index 
which facilitateo both the reduction and storage of task data in a sys- 
tematic fashion. (The classification is recorded in Columns NO of 
Form 07 for each task statement associ ced with the output being ana- 
lyzed. ) 



Definitions 



Task Cluster Category - A classification given a task statement 
which identifies it as one of a number of different tasks having 
certain general qualities in common. 



Decision rules : 



(a) the task statement clearly 
serves the process character- 
istic described by a single 
task clus ter category; o£ 

(b) a single cluster category set 
contains within it a task des- 
cription which matches the 
task statement (when there is 
doubt about a choice between 
two or more major categories); 

?S 

(c) it is determined no appropriate 
cluster category exists and a 
new one is established and 
given an initial label thought 
to best represent the general 
process characteristic implied 
by the task statement. 



Examples: Clarifying the Problem: includes "Specify 

the variables to be studied"; "identify con- 
taminating variables"; "review problem-related 
literature," etc. Procuring Professional 
Staff : includes "analyze work for competenc- 

ies required" ; "determine compe tencies avail- 
able; obtain approval to hire," etc. (See NO 
and QR Categories in Appendix 4 for complete 
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Discussion of Category Set NO Decision Rules 

This category but is designed to accept new clusters of tasks as 
they become apparent. In so doing it must be remembered that each set 
established will be considered mutually exclusive from any other. 
Occasionally, as with quality control tasks, different general process 
characteristics may be represented. It is therefore essential that all 
NO and QR sets be examined before determining a need for a new cluster. 

Note also that there is a break in the NO Category Set numbering 
which separates production tasks from those tasks which relate to manip- 
ulation of processes and/or operations. Should a new cluster category 
set be established, it must be listed within the appropriate subset. 
Production tasks are those pe rceived to relate to outputs which are 
essentially the object of project efforts, while process and/or oper- 
ations tasks are those which are perceived to provide resources and a 
quality environment for the project as a whole. (The primary use of 
the two subsets of clusters is as an index to existing clusters which 
facilitates coding efforts. Subsequent verification of the integrity 
of the items coded to each subset would permit analyses to be done on 
output vs. process /operat ions tasks. 



Category Set P: Self/Qthers Report for Tasks 



Purpose of the Classification 

The self/others classification for tasks permits identification 
of those tasks, reported necessary in the achievement of an output, 
which are performed by the interviewee, others, or the interviewee and 
others as a team. (The classification is recorded in Column P of Form 
07 for task statement associated with the output being analyzed.) 



Definition s 

Code Se lf/Others Report for Tasks 

1 Self (reported by interviewee as relevant to his own work on 
the output). 

2 Others (reported by interviewee as relevant to the work per- 
formed on the output by other project personnel, but not as 
performed by the interviewee) . 

3 Team (reported by interviewee as relevant to the combined 
efforts of his own work on the output and the concurrent work 
of others). 
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biricussion of Category Set P 

The coding represented by this category set must be made by the 
interviewer, or individual recapping audio-tapes of interview, on the 
basis of the interview protocol. Instructions to interviewers specify 
that it must be clear whether each task reported by the interviewee is 
performed essentially by himself, others, or himself and others as a 
team. 



Category Set QR: Primary Categories of Tasks 



Purpose of the Classification 

The purpose of this category set is to quantify the various tasks 
associated with achieving an output, including the various ways or 
levels of specificity in which they are expressed, and the frequencies 
with which they are cited. Such quantification of tasks performed for 
each output permits identification of the manner in which various tasks 
may distribute across outputs of differing classes, levelfe, foci, and 
functions, (The classification is recorded in Columns QR of Form 07 
for each task statement associated with the output being analyzed,) 



Definitions 



Primary Task Category - A basic classification given a discrete 
unit of work in which the classification reflects the fundamental 
and essential elements which mark that unit as a unique step in the 
accomplishment of an output, (See discussion section for elabora- 
tion of the elements of this definition ,) 

Decision rules : (a) the general behavior inherent 

in a task statement corresponds 
in level (included elements) , 
action, and target of that 
action to that reflected by a 
single primary task category 
listed in the existing sets 
(Sat QR) ; £r 

(b) the task, as expressed does not 
correspond as in (a), and i; 
added to the appropriate task 
cluster category (Sat NO) list- 
ing. 

Examples: ’^Negotiate quality issues with sponsor" is 

different from "negotiate contract"; "making 
a decision" is different from "determining 
key decision points," etc. These differences 
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are the has is for de tenuining the match of a 
statement to a primary category. (See Cate- 
gory Sets NO and QR in Appendix 4 for a com- 
plete listing of cluster and primary task 
categories es tab lished to date . ) 



Discussion of Category Set QR Decision Rules 

Time and space does not permit consideration of all possible prob- 
lems in the handling and classifying of "task** data. Indeed, the infi- 
nite variations in the level of detail possible to elicit from various 
respondents without risking interviewer biasing of the data being 
gained (inappropriate leading questions) suggests the most meaningful 
discussion should center on the principles involved . 

First, an elaboration of the term **task** seems appropriate. In 
content , a task is generally said to describe worker activi ty . I t is 
a unit of activity that deals with the methods, procedures, and tech- 
niques by which outputs are produced. A task is a discrete unit of 
work performed by an individual or individuals where there is a defi- 
nite beginning and ending within a limited period of time (albeit a 
reiterative instance of). It is not so finite as to constitute a 
single homogeneous action of the worker (e.g., **I pick up the pencil*'), 
but it is a composite of such homogeneous actions which describe mean- 
ingful units of work performed. A '*meaningf ul" unit is described as 
that level of detail it is possible to elicit from a given informant 
without the statement assuming a ''homogeneous action" characteristic. 

Given this definition , then , classification of raw task s tate- 
ments involves either matching it to an existing primary category in 
terms of its level and obvious inclusions, or failing in that, adding 
it to the appropriate category listing as a new primary category. The 
addition of an item must be done with care and in keeping with the 
practJ-cal aspects of meaningful overlap and redundancy of information. 
(Subsequent analyses of data, at given future points in time may permit 
col laps ing and rewording of some categories to eliminate unnecessary 
redundancies . ) 



Category Set S: Structure Categories of Enablers 



Purpose of the Classification 

The classification of enablers into structure categories permits 
the initial clustering of broad ranging enabler statements according 
to characteristics they have in com on. (The classification is recorded 
in Column S of Form 08 for each enal ier statement associated with the 
output being analyzed.) 

^ V o 
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De f i nl tlons 

Structure of an enabler - The characteristics of an enabler state- 
ment which identify it as representing a requisite knowledge » 
requisite process skill , or requisite sensitivity in the accomp- 
lishment of an output. 

Code Structure Category 

1 Knowledge - An instance of that which must be known or known 
about as a requisite to the successful accomplishment of an 
output • 

Decision rules: (a) the statement refers to some- 

thing that one needs to know in 
order to accomplish an output; 
or 

(b) the statement refers to some- 
thing that one needs to know 
how to do in order to accomp- 
lish an output. 

Examples: '*Know the objectives." 

"Know what teachers are asked to do." 

"Know the professional language . " 

"Know how to budget." 

"Know statistics." 

2 Skill - A specific ability, proficiency or expertness con- 
sidered requisite to the successful accomplishment of an out- 
put . 

Decision rules: (a) the statement refers to a 

process in which one must have 
some degree of ability in order 
to successfully accomplish an 
output . 

(b) the statement refers to an 

expertness which one must pos- 
sess in the application and/or 
utilization of resources or 
enabling knowledges and sensi- 
tivities brought to bear on 
achieving an output. 

Examples: "Ability to visualize the overall operation." 

"Ability to administer tests." 

"Ability to adapt to a situation." 

"Ability to exercise judgment and make deci- 
sions . " 

3 Sensitivity - A specific perceptiveness and responsiveness 
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considered requisite to the successful accomplishment of an 
output . 



Decision rules: 



(a) the statement refers to a need 
to have a sense or awareness of 

a phenomenon, condition or *'state 
of affairs** in the environment ; 
and/or 

(b) the s tatement implies, directly 
or indirectly, an awareness of 
the need to be responsive to 
such phenomena, conditions, or 
states; oic^ 

(c) the statement refers to an atti- 
tude one must have relative to 
an implied phenomenon, condi- 
tion, or state. 



Examples : "Sensi tive to the limitations of others .** 

*'Sense of reality in establishing goals.** 
**Awareness of alternative courses of action . 
*' Aware of the affective qualities of an out- 
put ." 

"Willingness to experiment or hypothesize.'* 
"Willingness to accept guidance . ** 



Discussion of Category Set S Decision Rules 

For present purposes, the category set includes three structure 
categories. The decision to add another may depend on the phenomenon 
upon which one chooses to focus, e.g., attitudes or personality charac- 
teristics as discrete from sensitivities. In any event, the parameters 
for classifying enablers within three structure categories have proven 
reasonably exhaustive to date. It must be noted that it is conceivable 
any enabler statement classified according to one structure category 
criterion (e.g., knowledge) could, in fact, have a skill and sensitiv- 
ity counterpart. In the mapping effort conducted, this possibility was 
hypothesized, but in keeping with the data-dependency criteria for the 
creation of a category it was not presumed to exist until it appeared* 
For that reason, there are similarities between knowledges, skills, and 
sensitivities in terms of the objects of their attention . The language 
and intent of a statement mus t , therefore, be clearly understood before 
classifying it as representing one of the In keeping with the 

logic of development of the detailed en'ibler sets (Set UV) , it is imper- 
ative that this classification (Set S) be made independently from the 
primary categoi fes included within each. 
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Catej^ory Set T; Self/Others Report for Enablers 



Purpose of the Classification 

The self/others classifications for enablers permits identifying 
the manner in which enablers distribute across people in accomplishing 
outputs of differing classes, levels, foci, and functions. (The clas- 
sification is recorded in Column T of Form 08 fur each enabler statement 
associated with the output being analyzed.) 



Def ini tions 



Code Self/Others Report for Enablers 

1 Self (reported by interviewee as relevant to his own work on 
the output) . 

2 Others (repot, i by interviewee as relevant to the work per- 
formed ou the output by other project personnel, but not as 
performed by the interviewee). 

3 Team (reported by interviewee as relevant to the combined 
efforts of his own work on the output and the concurrent work 
of others) . 



Discussion of Category Set T 

The coding represented by this category set must be made by the 
interviewer, or :.ae individual recapping audio-tapes of interview, on 
the basis of interview protocol. Instructions to interviewers specify 
that it must be clear whether the enabler reported by the interviewee 
is a requisite he must have in the performance of his work relative to 
an output, or one others must have in relation to him and the same 
output . 



Category Set UV; Primary Categories of Enablers 



Purpose of the Classification 

The purpose of this category set is to quantify the various ena- 
blers associated with achieving an output, including the various ways or 
levels of specificity in which they are expressed, and the frequencies 
with which they are cited. Such quantification of enablers permits 
identification of the manner in which various enablers m,ay distribute 
across outputs of differing classes, levels, foci, and functions. (The 
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classification is recorded in Columns UV of Form 06 for each enabler 
statement associated with the output being analyzed.) 



Definitions 



Enabler - That which provides the ability, means, opportunity, 
power or authority to successfully accomplish an output. For the 
present data enablers are ordered by knowledge , skill , and sensi - 
tivity categories (Category Set S) . 

Decision rules : (a) the statement refers to a knowl- 

edge, skill, or sensitivity 
(Category Set S) which matches 
the level or inclusions within 
each set Implied by an item; 
or 

(b) the statement does not meet the 
requirements of (a) and is added 
as an item to the listing within 
the major category set. (See 
next section for critical dis- 
cussion.) 

Examples: Knowledge oi project objectives is different 

from knowledge of similar efforts ; ^ skill in 
**wri ting** is different from ** communicating 
clearly**; sensitivity to **val^ systems** is 
different from sensitivity tor1Jf5|j|tential con- 
flicts of interest.** Such diff^ences are 
the basis for determining the exact match 
of a statement to a primary category. (See 
Category Sets S and UV for detailed listing 
of categories established to date. ) 



Discussion of Category Set UV Decision Rules 



% 



Of all the category sets which could be esFablished for a data 
collection effort, the enabler set is perhaps the most vulnerable *:o 
vagaries of reality, particularly in relation to knowledge factors . 

While other sets attend to factors which occur witliih the defined para- 
meters of educational RDD&E, enablers can originate^or Li derived from 
any world of experience. For that reason, the present enabler category 
set must be perceived to be in a formative, prototype"^ state of develop- 
ment and less refined than other sets. Items within the knowledge set, 
for example, begin by discriminating on the basis of the domains or 
disciplines of content Represented by a statement, with substantive 
meaning depending on the retrieval of the raw statements. Other items 
within the same set were explicated when they did not appear to repre- 
sent a content item ^o^f arfother discipline (e.g., English, theories of 
learning, computer technology, etc.). The decision to explicate items 
for addition to the listings may well depend on the locus of the effort. 

li' 
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Subsequent analysis of the present data would include content analysis 
of raw staterrents linked to, e.g., "Subjects primarily related to 
RDD 61 E," and the knowledge listing expanded accordingly. 

In the case of skills and sensitivities, the listings are more 
clearly substantive and directly represent the levels at which inform- 
ants were able to express them. In any event, the principle involved is 
to examine all primary categories (wi thin the appropriate structure 
category) and a raw statement in terms of: 

(a) the degree to which they most nearly mr>.tch; 

(b) the level of meaning (or utility) required of 

a category in order to attend to the objectives 
of the data collection effort.; 

(c) a sense of the number of variations within a 
variable that it is reasonable to as:;ume one 
can meaningfully handle. 



Category Set W; Output Completion Stage 



Purpose of the Classification 

This classification is made to permit identification of the degree 
to which data obtained relative to a given output is current in nature, 
i.e., whether it is generally retrospective, current, or projected data. 
This in turn permits one to control for "currency of perception" factors 
when sampling as well as to subsequently examine differences in retro- 
spective data as opposed to current or projected impressions . (The 
classification is recorded in Column W of Form 05B for each output iden- 
tified in a project.) 



Definitions 

Code Output Completion Stage 

1 Completed a long time ago (over one year ago). 

2 Completed some time ago (three to twelve months ago). 

3 Recently completed (within last three months). 

4 Currently in process (outputs having defined completion 
points; see Code 6 ), 
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6 On-going (generally continuous throughout project , e .g . , 

staff morale) . 



Discussion of Category Set W 

The definitions for these classifications generally seem self- 
explanatory, Codes 4 and 6 discriminate between those outputs currently 
being attended but which have defined beginning and ending points and 
those outputs whicl: are attended continuously throughout t’ e life of a 
project or effort. 



Category Set X; Project Role of Interviewee 



Purpose of the Classification 

The classification of each interviewee by the role with which he 
is most closely identified within a prciject permits an analysis of the 
manner in which work activities and work requirements of people and the 
outputs they produce, may vary across projects and project roles. (This 
classification is recorded as Item 47 in Questionnaire Form 03 fer each 
project staff member.) 



Definitions 



Code Project Roles 

1 Principal Investigator ( contractual responsibili ty) 

2 Project Director (if other than Principal Inves tigator) 

3 Top (or intermediate-level) Project Manager or Coordinator 
(other than P.I. or P.D.) 

4 First-line Supervisor (supervising three or more project 
staff members) 

5 Member , Professional Staf f (of the project) 

6 Member, Clerical Support Staff (of the project) 

7 Member, Tech*".ical Support Staff (of the project) 

8 Advisor or Consultant (to the project) 

9 Other 



Discussion of Category Set X 

These categories seem essentially self-explanatory. However, at the 
closing out of an interview contact with an i^formapt, the interviewer 
must carefully discuss with him each of‘' the project role categories. 
Determination of the classification to be given the informant must be 
based on which category best characterizes the primary responsibilities 
of his role, keeping in mind the manner in which the role is viewed by 
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oLfu^rs. Ihe interviewer may assist in the classification throuji^h expres- 
sing his own impressions of the informant’s role (based on insigh.s 
from interview) but, in any event, the classification ultimate!;' made 
must be agreed to as best reflecting the rcle held. 



Category Set Y: Project Focus 



Purpose of the Classification 

The classification of a project according to its primary focus 
permi ts^^examina^'ion of the manner in whi :h outputs having separate 
character classxfications distribute ac*.^ projects of differing foci. 
This clas5jiticat ion is reported and recorded for each output of a pro- 
ject in Column Y of Form 05B. 



Definitions 



Code Focus 

1 Research - An outcome of project effort in which the primary 
goal of the effort is viewed as the achievement of an idea, 
principle, theory, or law that can be shown to be generaliz- 
able through empirical verification. 

Decision rules; (a) the primary outcome of the pro- 

ject has the characteristic cf 
having been conceived and empir- 
ically tested by the project 
through the process generating 
it; and 

(b) the outcome has the characteris- 
tic of an idea, principle, etc., 
derived from data, or is a report 
of those things, having utility 
or application beyond the con- 
text in which it was generated. 

Examples ; Nature of reading; conceptual papers (empir- 
ically derived and tested) ; experimental 
(research) data; data analysis and interpre- 
tation (research data) . 

2 Development - An outcoiae of project effort in which the pri- 
mary goal of the effort is viewed as the achievement of a 
reliable technology, that is, procedures, materials , hardware, 
and organizational frameworks that when applied can bring 
about a desired outcome or perform a defined operation. 



a?PI:NDIX 5 (contM.) 



Code 

3 



Decision rules: (a) the outcome has the essential 

character is tic of providing a 
framework for accomplishing a 
specified end; and 
(b) the outcome has thr essential 

characteristic of u ^.ng explicit 
an end which may be a^hed as 
r'. result of its application . 

Examples: An instructional system; data management sys- 

tem; total observation system. 



Focus 



Diffusion - An outcome of project effort in which the primary 
goal of the effort is viewed as bringing about the adoption 
and utilization of knowledge , information and/or technology . 

Decision rules : (a) the outcome has the character- 

istic of transmitting knowledge, 
technology, and/or information ; 
and 

(b) the outcome is one of others 

having adopted and/or utilized 
information or technology within 
certain specified parameters. 

Examples: Commeicial edition of a catalog of objectives; 

popular magazine publication supporting a 
program; final report (whei. servin', account- 
ability and not meeting other-focus criteria). 

Evaluation - An outcome of project effort in which the primary 
goal of the effort is viewed rr the production of trustworthy 
infoi^ation regarding a phenomenon which occurs in a context or 
environment over which the user of the information expects to 
exercise influence or about which he expects to make decisions. 



Decision rules : 


(a) 


the outcome hes the characteris- 
tic of having been legitimized 
through the p::’ocesses generating 
it; and 




(b) 


the outcome has the characteris- 
tic of data, or a report of that 
data, having utility or applica- 
tion within the context of its 
generation. 



Examples: Statistical and reference reports; performance 

testing data; criterion referenced test item 
poo 1 ; normative data . 
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Discussion uf Category Set Y Decision Rules 

This classification is relatively straightforward as long as a pro- 
ject or defined operation has a single focus in relation to contract or 
obligation. However, there are efforts in which there are clearly mul - 
tiple contractual ob ligations , e.g. , some obligations require the pro- 
vision of evaluative data, and at the same time, the development of an 

evaluative technology that can be used in other contexts. In other pro- 

jects multiple thrusts of research for the sake of new knowledge is 
coupled with the development of new means to achieve special learning 
outcomes. In such cases, the decision for providing the project focus 
classification to a project is based on the determination of a single, 
primary or superordinate focus of the project. This determination may 

be extremely difficult, at times, and is guided by the following consid- 

erations : 

(a) the elements used to define the total effort as 

a "project," as mutually agreed upon with project 
personnel ; 

(b) the relative frequency the character classifications 

of focal outputs , such character classifications inter- 
preted as the outputs of specific foci; 

(c) in the absence of discrimination on the basis of (a) 
and/or (b) , the relacive weighting of resources and 
perceived priorities of staff; 

(d) the classification ultimately ^ keeping with the 
terms of the definitions of each proj ect focus 
described ab ove . 

It must be noted that this classification is, in some cases, a 
forced one serving a very gross purpose. The reader is referred to 
Category Set Z in which the classification scheme deliberately accounts 
for multiple foci within projects. The classification in the present 
instance is closely tied to site selection criteria and may, in fact, 
influence either the selection of sites or the parameters used to define 
the "project." For example, in an on-going operation involving any or 
all of RDD&E and in the absence of other criteria which may define it, 
one may choose to describe the "project" parameters to be studied in 
terms of a single focus such as the Evaluation effort. 

In any event. Category Set Y requires identification of a single, 
primary focus. When the analysis made possible by this classification 
is not desired, Category Set Z makes possible a similar analysis based 
strictly on any separate focal points of project efforts. 
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Category Set Z; Chara cter of Supported Focal Output 



Purpose of the Classification 

The classification of each output according to the character of 
the focal output for which it is accomplished is to provide a means 
of identifying the manner in which the character of those outputs vary 
within focal outputs having separate character classifications. 



Definitions 

Code Character of Supported Focal Output 



1 Knowledge (See Category Set C) 



A Information 



Discussion of Category Set Z Coding 

This classification is relatively straightforward in that it is 
merely transcribing the Category Set _C code given the focal output to 
which the output in question contributes. Where the "Z** code is being 
considered for a focal output, its own *'C” code is used. (See Category 
Set C for definitions and decision rules.) In some instances an output 
may contribute, in different ways, to more than one focal output. When 
this is the case, a judgment must be made regarding which focal output 
niost dependent for its accomplishment on the output in question. 
Output maps and other interview data described elsewhere in this volume 
provide information relative to this dependency factor. 
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Technol ogy 
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II 



II II 
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Implementation 



II 



II 



II II 
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Computer Program Listing 

RDDE7 

ISO 

LIMIT 

HEADER 

MEAN4 

PRECON 

TABi^EF 

SORT 
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RDDE7 



Computer Prof’ram Listing 



0S3 FORTRAN VERSION 2.1 02/28/72 1718 

PROGRAM R00E7 ‘ R07 

C PROGRAM PERFORMS SELECTED SORTS ANO TABLES SELECTED VARIABLES R07 

C «D7 

C0HM0N/0ATA/NN(7) ,1NP0(7,2) »LIH(23L) * R07 

C ARRAY NN DEFINES NO. OF VARIABLES AT EACH LEVEL R07 

0ATA(NN=35,47,17,99,12,l?,12l R07 

-ARRAY INPO defines LUN NUMBERS FOR DATA SETS. LEVELS lf2|3»ANO U R07 
USE UNIQUE DATA SETS. LEVELS 5, 6, ANO 7 USE TWO DATA SETS EACH. R07 

FIRST SET ORDERED BY OUTPUT NUMBER. WITHIN PERSONNEL NUMBER, WITHIN . . R07 
PROJECT NUMBER(J=1) , ANO SECOND SET BY PERSONNEL NUMBER WITHIN OUT- R07 
PUT NUMBER WITHIN PROJECT NUMBER(J=2). R07 

PROGRAM CONTROL INFO. ON LUN L5 R07 

OUTPUT ON LUN L6 R07 

MONITOR ON LUN 47 R07 

0ATA( ( (INP0(I,J) ,1 = 1 ,7) , J = l,2) =34,35,36j37j.38,39,40x R07 

134, 35, 36, 37, 31, 32, ?3) R07 

-DEFINES MAX. VAR. VALUES FOR LEVEL 1 R07 

OAT A ( (LIM (I), 1=1,35) =25,4,6,9,4,5,4, 12 ,71, 8, 10,6, 6,3 ,9,99 ,99, 62,67 R07 

1.7. 7. 9. 9. 13.12.10.4.25.10.5.5.4.3.5.3) R07 

-DEFINES MAX. VALUES LEVEL 2 RD7 

_OATA( (LIM(I)j I = 36,82) = 25,2C,100,2t3,.9u‘*j5,12,12j7j35,25,22,10,17, R07 

122,17,25, 19,75,40,99,50,81,60,7,8,8,9,210,210,210,210,210,210,210, R07 

2210,210,210,210,210,210,210,210,10,10) R07 

-DEFINES MAX. VALUES LEVEL 3 R07 

DATA( (LIM (I), 1=83, 99) =25, 210, 576,6,3,3,4,18,16,14,14,52, 336,4,4, R07 

14.3) R07 

-DEFINES MAX. VALUES LEVEL 4 R07 

OATA( (LIM (I), 1=10 0,1 98) =25, 20, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, R07 

18.8.8.8.8.8.8.8.8.8.8.8.8.8.8.8.8.8.8.8.8.8.8.8.8.8.8.0. 8.8.8.8.8, R07 

23.8.8.8.8.8.8.8.8.0. 0.8.8.8.8.8.8.8.8.8.8.8.8.8.8.8.8.8.8.8.8.8.8, R07 

38.8.8.8.8.8.8.6.8.8.0. 8.30, _ R07 

-DEFINES MAX. VALUES LEVEL 5 R0.7 

„OATA( (LIM(I) , 1 = 199, 2 10) =25. 20, 2 10, 65, 3,3, 4 , 52, 336 , 2, 3, 46) RO^ 

-DEFINES MAX. VALUES LEVEL 6 ' R07 

OATA( (LIM (I), 1=211, 222) =25, 20, 210, 71, 3, 3, 4, 52, 336, 33, 3, 32) R07 

-DEFINES MAX. VALUES LEVEL 7 R07 

OATA( (LIM (I), 1=223, 234) =25, 20, 2 10, 65, 3, 3, 4, 52, 336, 3, 3, 58) R07 

-IVARIARRAY STORES VARIABLE ID NUMBERS FOR REAL SORT R07 

_LEVELIARRAY STORES WITHIN VARIAB LE LEVEL NU MBER FOR REAL SORT R07 

DIMENSION IVAR(50) ,LEVEL(5'0) R07 

C DEFINE DATA RECORD SCRATCH LUN FOR TABLES _ R07 

ISCR=44 R07 

C WRITE HEADER ON MONITOR LUN . _ _ _ R07 

WRITE(47,10) - - - - 

C INITIALIZE COUNTERS, FLAGS RO^. 

LSORT=iNOS=LSORTl = 0 R07 

100 ISKP=1 R07 

NTBL=ICRS1=0 R07 

C READ ALPHANUMERIC TABLE HEADER R07 

CALL HEAOER(NTBL,LSJRT,NOS,l) R07 

C---— increment data set counter R07 

N0S=N0S*1 R07 

C WRITE DATA SET NUMBER ON OUTPUT LUN R07 

WRITE(46, 16IN0S R07 

C-- READ CONTROL INFORMATION FOR NEXT LEVEL TO BE SORTED R07 

110 REA0(45,11)LS0RT1,MS0RT,LUNTBL,ICRS, I0P4 R07 

IF<E0F(45))ST0P _ . R07 

120 LSORT=LSORTl ' " ' ' ' ' ' “ R07 

C IF NEW DATA SCf (LSORT=0) GO TO lOC R07 

IF(LSORT.EQ.O )GO TO 100 R07 



ERIC 
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28 
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37 

38 
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RDDE7-2 



0S3 FORTRAN VFRSION 2.1 RD0E7 02/28/72 1718 

c define scratch Input lun for this level ro7 

IF( ICRS.EO. 2) ICRSl ‘1 R07 6? 

INPS = A2 .. . „ R07 63 

IF(ICRS.LE.1.AND.LS0RT.L£.3>INPS«41 RD7 6i. 

IF( INPS.EQ. Rl) ICRS1=0 R07 66 

IF(ICRS1.EQ.1.ANO.LSORT.GT.3) INPS=‘.3 R07 66 

C IF SORT LEVEL >3 GO TO 130 R07 67 

IFILSORT.GT .3»GO TO 130 RD7 66 

. ,.C DEFINE scratch OUTPUT LUN FOR .THIS .LEVEI R07 69 

ISKP=1 RD7 70 

I0UT=42 RD7 71 

IFILSORT.EQ. 1H0UT=41 RD7 72 

IF{ ICRS.EQ. 2) IOUT = 43 R07 73 

130 NVARsO R07 74 

C --GO TO 150 IF ^NOMINAL# SORT RD7 75 

IF(HSORT.LE.O)GO TO 150 ~ ~ RD7 76 

C READ NO. OF VAP/LE VELS, TO SORT; THEN VARIABLE/LEVEL COMBINATIONS RD7 77 

REA0(45f 12) NVAR, ( IVARm f LEVEL (I) f I = lf NVAR) R07 78 

C CALL SORTING SUBROUTINE RD7 79 

150 CALL SORT (INPOCLSORT ,ISKP> fINPS, lOUT fISCRfNVAR, I VAR, LEVEL fLSORT, R07 60 



C IF NO RECORDS MEET CRITERION GO TO 160 RD7 82 

IFCNREC.LE. 0) GO TO 160 RD7 83 

C WRITE MONITOR INFORMATION RD7 84 

WRITE(47,13)NDS,LS0RT,ICRS,NVAR,NREC RD7 85 

C IF TABLES TO BE MADE CALL TABLING ROUTINE RD7 86 

IF(LUNTBL.GT.O)CALL T ABLEF ( ISCR , LSORT , LUNTBL , NT.BL ,NDS.) RD7 87 

C- IF MEANS-PERCENTAGES ANALYSIS DESIRED CALL ROUTINE RD7 88 

IF(IOP4.GT.O. AND. LSORT. EQ.4)CALL ME A N4 ( ISCR, L SORT , NOS > R07 89 

GO TO 110 RD7 90 

C WRITE /NO RECORD/ INFORMATION ON OUTPUT LUN AND MONITOR LUN RD7 91 

160 WRITE(46,14)N0S, LSORT, ICRS, NVAR R07 9? 

MRITE(47, 14)H0S, LSORT, ICRS, NVAR RD7 93 

C IF SORT LEVEL > 3 GD TO 110 RD7 94 

IF(LS0RT.GT.3)G0 TO 110 RD7 95 

C READ NEXT HEADER RECORD FOR SORT LEVEL RD7 96 

170 READ(45,11)LS0RT1,MS0RT, LUNTBL, ICRS RD7 97 

IF(E0F(45) ) STOP RD7 98 

C IF NEW SORT LEVEL <= /NO RECORD/ SORT LEVEL GO TO 120 RD7 99 

IFfLSORTl.LE.LSORTlGO TO 120 RD7100 

NVARsO RD7101 

C IF NOMINAL SORT GO TO 180 RD7102 

IFIMSORT .LE.Q)G0 TO 180 R07103 

C READ NUMBER OF VAR. /LEVELS, VARIABLE/LEVEL COMBINATIONS RD7104 

READ(45,12)NVAP, (IVAR(I) , LEVEL (I) ,1=1, NVAR) . RD7105 

C WRITE MONITOR INFORMATION ON OUTPUT LUN AND MONITOR' lUn RD7106 

180 WRITE (46, 15>N0S,LS0RT1, NVAR R07107 

WRITE(47,15)NDS,LS0RT1,NVAR RO7108 

GO TO 170 RD7109 

10 FORMAT(/l DATA SET/,2X,/ SORT LEVEL /,2X,/ CRC / ,2X , / VARI ABI.E R07110 



14 FORMAT(/l NO RECORDS, DATA SET / , 15 , / , /O / , 14X , /SORT LEVEL /,I6, R07115 

1/, /0/,14X ,/CRS /,I6,/,/0/,14X,/NO. OF VARIABLE/LEVELS /,IB> R07116 

15 FORMAT(/0 DELETED, DATA SET /,I6,/ SORT LEVEL /,I5,/ NO. OF VARI/ RD7117 
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13 


FORTRftN VERSION 2.1 


02/26/72 1718 








FUNCTION I30( II 9 JJ*KK, IR, 


IC) 


ISO 


1 


c 




FOR CONVERTING if2tOR 3 OIMENSICNS INTO 


ISO 


2 


c 


... .<V. SINGLE DIMENSION WITH ZERO _AS_JHE ..ORIGIN ... 


..ISO 


3 


c 


IIIROH SUBSCRIPT VALUE 




ISO 


4 


c 


JJiCOLUMN SUBSCRIPT 




ISO 


5 


c 


KKILEVEL SUBSCRIPT 




ISO 


6 


c 


IRlNUMBER OF ROWS 




ISO 


7 


c 


ICiNUMBER OF COL. 




ISO 


• 8 


c 






ISO 


9 




IS0 = KK*IR*IC<-JJ»IR>II>1 




ISO 


10 




RETURN 




ISO 


11 




ENO 




ISO 


12 



NO, ERRORS FOR ISO . , 

P 00052 C 00000 0 00000 
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LIMIT 



0S3 FORTRAN VERSION 2.1 ^ZtZhtlZ 1719 





FUNCTION LIMIT(I.J) 


LIM 


1 




RETURNS NO. OF CATEGORIES FOR LEVEL I. VARIABLE J 


LIM 


2 


c 


. _ ARRAY ILMINO. OF VARIABLES AT EACH LEVEL 


LIM 


3 


c 


ARRAY IQIOUMMY 


LIM 




c 


array LIMINO. of CATEGORIES FOR EACH VARIABLE 


LIH 


S 




COMMON/OATA/ILM(7) .lOa*.) .LIM(234) 


LIM 


6 


c SET 00 LOOP LIMIT 


LIM 


1 




111=1-1 


LIM 


6 


c- 


-“—-SET COUNTER TO 7ERO 


LIM 


.3 




ILIM=0 


LIM 


10 


c- 


IF LEVEL 1 GO TO 120 


LIM 


ii 




IF(II1.LE.O)CO TO 120 


LIM 


12 


c- 


COMPUTE NO. OF VAR. IN LEVELS 1 TO I-l 


LIM 


13 




00 110 II«1,II1 


LIM 


14 




110_ILIM=ILIM4ILM(II) 


LIM 


15 


c- 


ftOO VAR. NO., GIVES SUBSCRIPT IN ARRAY LIN 


LIM 


16 




120 ILIM=ILIM*J 


LIM 


17 


c- 


— -DETERMINE NO. OF CAT. 


LIM 


IS 




LIMIT«LIM<ILIM) 


LIM 


19 




RETURN 


LIM 


20 




END 


LIM 


21 



NO ERRORS FOR LIMIT 
P 00067 C 00000 0 00377 
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HEADER 



0S3 FORTRAN VERSION 2,1 02/28/72 1718 





subroutine HEAOERINTBLtLSORTtNOStlROI . 


- .. HOR 


1 


c- 


READ HEADER ARRAY IF IRO > 0 


HDR 


2 


X* 


— — -MRITF TABLE HEAQFR IF IRO <s 0 


HDR 


.3 


c< 


NT0LI TABLE NUHBER COUNTER 


HDR 


4 


c 


LSORTILEVEL OF SORT 


HDR 


5 


c 


NOSlOATA SET NUHBER 


HOR 


6 


c 


IROIREAO OR WRITE FLAG 


HDR 


7 


c- 


——ARRAY 10 STORES ALPHA NUHERIC HEADER FOR PRINTER OUTPUT 


HOR 






.DIMENSION IDC3CI 


hdr_ 




C— — -GO TO 100 TO WRITE HEADER 


HOR 


10 




IFdRO.LE.OlCO TO 100 


HOR 


11 




REA0(45tlll 10' 


HOR 


12 




RETURN . 


.. HDR 


13 




100 WRlTE(46tlOIIOtNOStLSORTtNTBL 


HOR 


14 




1.0_£ORMA T ( FI / , 30 A4 , / , / 0 ,.0AtA . SET UJ 1 SORI_WE VEL_£,I4j 4X i * _ 


HDR. 


15 




ITABLE NUHBER FtI6/l 


HOR 


. 0 




11 FORMAT(30A4) . _ 


HOR 


17 




RETURN 


HOR 


18 




END , 


HOR 


19 



no' ERRORS FOR HEADER 
P 00153 C 00000 0 00000 
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MEAN4 



0S3 FORTi^AN VERSION 2.1 MEAN** 02/28/72 1718 

C WRITE NO. OF RECORDS, MEANS, S .0. #S, PERCENTAGES, NO. OF NO RESP0NSESMN4 61 



WRITE U6, 10 )YN, (1 , SUMI 1 ) , SUM2 ( I) ,XN(Z) ,PCT ( I ) , XNR (I ) , 1=1,95) HN4 62 

.. .10 FORMAT(#0 FORM 04 SUMMARY#,/, YO TOTAL NUMBER OF RECORDS # ,F 9. 0 , . MN4 63 

l/,#0 MEAN, S.O., PERCENT ZERO RESPONSES, PERCENT RESPONSES >3, #, MN4 64 

2AANO NUMBER OF NO RESPONSES#, ///, 95 ( Z5 ,4F10 . 3, F8 . 0//) ) MN4 65 

return HN4 66 

END NN4 67 



NO ERRORS FOR NE~AM4 ' 

P 02534 C OOOOO 0 QOOOQ 
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MEAN/t- 



0S3 FORT^ftN VERSION 2.1 02/26/72 1716 

SUBROUTINE HE AN^t (LUN»LSGRT i NOS) 

COMPUTES MEANS AND S.D./S; PERCENT ZERO RESPONSES? AND PERCENT >3 

RESPONSES FOR 95 ITEMS AT LEVEL..*! 

LUNILUN NO. FOR INPUT RECORDS 
LSORTISORT LEVEL 
NOSIOATA SET NUMBER 

DIMENSION SUM(95) ,SUM2(95) »XN(95) ,PCT(95) »JVAR(99) |XnR(95) 

NTBL=l 

C WRITE OUTPUT ID ARRAY . 

CALL HEADER(NTDLtLSORTfNDS>0) 

REWIND LUN 

C INITIALIZE VARIABLES 

YN=0. 

DO 90 1=1,95 

90 SUM(I)=SUM2(I ) = XNm=XNRaJ*PCTJI)=0, 

c read OATA RECORD 

100 READ(LUN) JVAR 

C IF EOF GO TO 135 

IF(EOF(LUN> )GO TO 135 
C INCREMENT RECORD COUNTER 

.YN=YN*1. 

DO 130 1=5,98 

C COMPUTE SUBSCRIPT 

<=I-4 

C INCREMENT NO RESPONSE COUNTER IF JVAR(I)>7 .. _ __ _ 

IF(JVARd) .GT.7)XNR(K) = XNR(K) ♦!. 

C IF NO RESPONSE GO_TO 130 

IFCJVARm .GT. ’)GO TO 130 

C ACCUMULATE SUM ... 

SUM(K)=SUM(K» ♦JVAP'I) 

C ACCUMULATE SUM OF SQUARES . 

SUM2(K)=SUM2(K)>JVAR(I) •JVAR(I) 

Cr-r-r:INCREMENT_ ZERO COUNTER IF_JVAR,<*0. 

IF(JVAR(I).LE.0)XN(K)=XN(io^l. 

C INCREMENT >3 COUNTER IF JVAR IS 4, 5, 6, 7 

IF(JVARd) .GT.3.AN0. JVAR(I) .LT . 6 ) PCT ( K) =PCT ( K) ♦! . 

130 CONTINUE . 

SUM(95)=SUM(95)>JVAR(99) 

SUM2(95)=SUM2(95) ♦ JVAR (99 )• JVAR (99) ^ 

I F( JVAR (99) .LT.1)XNR(95)=XNR(95)>1. 

GO TO 100 .. . 

C COMPUTE MEANS, S. D. AS, PERCENT ACES 

135 DO 140 K=l,94 

c compute no. nonzero responses 

ZN=YN-XNR(K) 

c --— compute S.O. 

SUM2(K)=SQRTF((SUM2(K)-SUM(K)*SUM(K)/ZN)/(ZN-1.) ) 

c compute MEAN 

SUM(K)=SUM(K) /ZN 
C COMPUTE PERCENT >3 

PCT(K)=PCT(K)/ZN*100. 

C-.'-'-. COMPUTE PERCENT ZERO RESPONSES 
XN(K) =XN(K) /ZN*100. 

140 CONTINUE 

C COMPUTE NO.RESPTSES FOR VAR. 95 

ZN=YN-XNR(95) 

C COMPUTE S.O 

SUM2(95)=SQRTF( (SUM2 (95) *SUH (95) *SUM (95) /ZN) / (ZN*1.) ) 

C-— -COMPUTE MEAN 

SUM(95)=SUM(95)/ZN 



MN4 1 
MN', 2 

._MN4_.3. 

HN4 4 
MN4 5 
MN4 6 
MN4 7 
MN4 8 
.-.MN4, .9 
MN4 10 
MN4 11 
MN4 12 
MN4 13 
MN4 14 
..MN4 .15 
MN4 16 
MN4 17 
MN4 18 
MN4 19 
MN4 20 
_MN4__21 
MN4 22 
MN4 2.5 
MN4 2-* 
MN4 25 
MN4 26 
_MN4 27 
MN4 28 
MN4 29 
MN4 30 
MN4 31 
MN4 32 
_MN4_33 
MN4 34 
MN4 35 
MN4 36 
MN4 37 
MN4 38 
_MN4 39 
MN4 40 
MN4 41 
MN4 42 
MN4 43 
MN4 44 
MN4 45 
■ MN4 46 
MN4 47 
HN4 48 
HN4 49 
MN4 50 
_MN4 51 
’ MN4 52 
HN4 53 
MN4 54 
HN4 55 
HN4 56 
MN4 57 
’ mN4 58 
MN4 59 
MN4 60 
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PRECON 



033 FORTRAN VERSION 2.1 PRECON 13j8 

C ENABLER ON RECAP PACE 

)0 I«l,20 

IF<..VAR<l,I).LE.iJ.ANO.I. GT.D.OO.TO.BLCi, . 

N2=N2^1 
00 53j J«1>3 
<=Jt9 

IF( JVAR(J|I).CT.NAX(K))MAX(K)=JVAR( Jil) 

530 <VAR()OsJVAR(J»I) 

C WRITE OUTPUT RECORD 

WRITE (lOUT) (<VAR(<) iK-i.12) 

540 CONTINUE 
■30 TO 510 
550 ENOFILE lOUT 

WRITE (61 1 17) LEVEL I N1|N2 1 (MAX( I) , 1 = 1 • 12) 

17 fORMAT(3I5,/,»CX12I5) 

30 TO 90 

C— --CONVERT LEVEL 6 AND FIND MAX. VALUES 
600 30 605 I>1»12 

605 WAX(I)=0 • 

W1»N2»0 

_610 REA0(INP,18) (IVAR(I) |I = 1|9) , (JJVARIUjIOxUrlf 3> »!i.=Ja^JUL 

18 FORMAT (31 3 1 12 1311112114120(12111112) ) 

IF(EOFdNP) )GO TO 65u 

N1=N1^1 
00 62u I«l,9 

IF(IVARd) .GT.MAX(I) ) MAX (I ) = 1 V AR (I ) 

62q_<VAR(I)«IVAR(I) ; 

00 640' 1 = 1,20 

IF(JVAR(lyI).LE.0.ANO.l.GT.l)GO TO 640 

N2=N2«1 

00 630 J=l,3 

<»J*9 

IF ( JVAR(Uj I ) . GT. MAX ( K ) ) MAX (KJL=_JV AR ( JjJ.) 

~630 kvAR(K)«JVAR( J, I) 

WRITEdOUT) (KVAR(K) yK=l,12) _ _ 

640 CONTINUE 
GO TO 610 
650 ENOFILE lOUT 

_ _ WRITE(61,17)LEVELiN1jN2, (MAX(IJ_,_I=1»12J 

GO TO 90 ~ 

END 



PREl 21 
PRE122 

..pr:i2o 

PRE,12‘. 
PEG 125 
PR£126 
PREl 27 
PRE12ft 
_.PRE122_... 
PRt 1 iu 
PRElol 
PRE102 
PRLl 23 
PREl OH 
_P.RE135 
PRE136 
PREl 37 
PREl 3() 
PREl 39 
PRE14J 
__PRE141._ 
PREl 42 
PREl 43 
PRE1h4 
PREl 45 
PREl4b 

_P.RE147 

PREl 48 
PREl 49 
PREl 5:1 
'’REIBI 
PREl 52 
..PRE1.5 3 
PREl 54 
PREl 55 
PRE156 
PRE157 
PREl 56 
_ PREl.59_. 
PREl 64 
PREl 61 



NO ERRORS FOR PRECON 
P 02305 C 0 lido 0 00122 
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0S3 FORTRAN ViPSION 2.1 PRECON u2/ia/72 13uS 
10 22. r = i,<*7 
IS=IS2(I) 

IF ( iVARdS) .LE.MftX (U)GO TO 220. .. 

^ax(I)=IVAR(IS) 

220 <\^AR(I) = IVAR(IS) 

WRITE(IOUT) (KVAR(I) , 

;0 TO 210 
250 cNDFiLE TOUT 

WRITE (61i 12)LEV£L|N1, ('1AX (II., L=1,<*7I . . . 

iO TO 90 

C CONVERT LEVEL 3 ANO FIND MAX. VALUES 

3u0 10 305 1=1,17 
3d5 MAX(I)=0 
W1 = 0 

310 REAO( INP, 14) ( IVAR{ I) ,1=1,17) 

14 F0RMAT(3I3,4I1,4I3,12,I4,4I1) 

IF(EOFdNP) IGO TO 350 
U=N1^1 

30 32u 1=1,17 

IF( IVARd) .GT .MAX(I) )HAX(l)=IVARd) 

320 CONTINUE 

WRITE d OUT) (I VAR d) ,1=1,17) 

GO TO 310 

350 ENOFILE lOUT 

WRITE(6l,12)LEVEL,Nl,(MAXd),I = l,l7) 

GO TO 90 

C- CONVERT LEVEL. 4_AN0_F.I.N0._MAX_._yjigJES 

40u )0 405 1=1,99 
4J5 1AXd)=0 
91 = 0 

410 REAOdNP, 15) dVARd) ,1 = 1,99) 

15 F0RMAT(2I3,2I2,94I1, 12) 

IF(EOFdNP) )Gp TO 45J 

NlrNl^l 
GO 420 1=1,99 

IF (MAX (I) .LE. IVARd) ) MAX ( I ) =I V AR( I ) 

420 CONTINUE 

WRITE(IOUT) (IVARd) ,1=1,99) 

GO TO 410 

450 ENOFILE IOUT ' ~ . 

WRITE(61,12)LEVEL,Nl,(MAXd),I = i,99) 

GO TO 90 

C CONVERT AND EXPAND LEVELS 5 OR T AND FIND MAX. VALUES 

C INITIALIZE MAX ARRAY 

5J0 GO 5G5 1=1,12 

505 1AX(i)=0 

C INITIALIZE RECORDS REAO(Nl) AND RECORDS PROOUCEO (N2) COUNTERS 

Ni. = N2 = 0 

C HEADER INFO IN IVAR, STANDARDS OR LNABLLR DATA FROM SAME 

C RECAP PAGE IN JVAR 

510 REAOdNP, 16) (IVARd) ,1 = 1,9) , ( ( JVAR ( J , K ) , J=1 , 3) , K=1 , 20) 

16 F0RMAT(3I3, 12, 311,12,14, 2a(2Il,i2) ) 

IF(EOFdNP) )GO TO 550 

C INCREMENT RECORDS READ COUNTER 

H1=N1*1 

C FIND HEADER MAX. VALUES 

GO 523 1=1,9 _ __ 

IFCIVARd) .GT .MAx'd) )MAxT'n=IVAR7'i) 

520 <VAR( n siVARd) 

C FINO MAX. values FOR VARIABLES 10,11,12 FOR EACH STANOARO OR 



61 

6C 

FRl 63 
P^f. 

b‘i 

PRC 66 
PRt 6/ 
P^6 6ft 
PRC 69 
P K c 7 j 
PRC 71 
PRZ 7d 
PRE 7 3 
PRC 74 



PRE 76 
PKE 77 
PRE 78 
PRE 79 
PRE 8J 

PREL>.8i^ 

PRE a? 
PRE 83 
PRE 8*4 
PRE 85 
PRE 86 

P.Rc.J.7_ 

PRE 8B 
PRE 69 
PRE 9u 
PRE .91 
PRE 92 
„-.PRE 91^ 
PRE 94 
PRE 95 . 
PRE 96 
PRE 97 
PRE 98 

PRE .99 

PREl uu 
PREiri 
PRE102 
PRE103 
PREl‘w4 

46 

’PRElGb • 
PRE1Q7 
PREl 08 
PRE1j9 ' 
PREl 13 
PREll.L^ 

p’reiie 

PREl 13 
PREl 14 
rREll5 
PRE116 

P?tll7_.. 

“ PREl 18 ” 
PRE119 
PRE12U 
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0S3 FORTRAN VERSION 2.1 a2/16/72 133S 

PROGRAM PRECON PRE i 

PROGRAM CONVERTS DATA FOR A SPECIFIC LEVEL FROM BCD TO BINARY PRE ^ 

FOR USE IN PROGRAM ROOE7 _ PRE ...3.. 

IVARI INPUT ARRAY FOR LtVtLS i f 2 , 3 , 4 , , 6 , 7 PRt <4 

JVARIIMPUT ARRAY FOR LEVELS 5»6,7 PRE 5 

RVARIOUTPUT ARRAY FOR LEVELS l,Z,b,b,7 PRE 6 

MAXISTORAGE ARRAY FOR MAX. VALUES CF EACH VARIABLE PRE 7 

CONTROL information ON LUN 4Si MAX VALUES PRINTED ON LUN tl PRE fl 

DIMENSION IVAR1135) , JVAR(3,20) . KVAR ( 47 ) . MAX ( 99) PRE. _ 9 

COMMOM/OAIA/ISK35) ,IS2(47) PRE 1. 

ISlIARRAY defines VARIAuLcS TO 3£ HRITTEN AS BINARY SORT RECORD PRE U 

FOR LEVEL I PRE Id 

QATA( (ISl(I) ,1 = 1. 35)=1.2|3,4,6i7,S«9.1[j|ll«l2»13.14.24.25,29,3ij, PRE li 

133, 34,37, 30,41, 42, <*3,46, 49, 51, 93, 59, 57, 59, 61, 63,64, 65) PRE 14 

— --IS2IARRAY DEFINES VARIABLES FOR LEVEL 2 PRE.. 15 

OATA( (IS2(I) , I=1,h7)=1,E, 3,4,5,0,7,0,9,13,14,22,23,24,25,26,27 ,20, PRE 16 

129,33,31,32,33,34,64,77,1^9,113,111,112,113,114,115,116, 117,.l8, PRE 17 

2119,123,121,122,123,124,125,126,127,134,135) PRE It) 

READ LEVEL TO BE CONVERTED, INPUT LUN, OUTPUT LUN PRE 19 

90 REA0(45,13)LEVEL,INP,I0UT PRE 2o 

„10 FORMAT (312) JPfiE.E? 

IF(E0F(45))St6f fRE 22 

REMIND INP PRE 2J 

REMIND lOUT PRE 24 

. GO T0(100,2jd,330,40j,5u0v6ca,5-3)LEVEL _ PRE 2;5 

CONVERT LEVEL 1 AND PRINT MAX. VALUES PRE 26 

INITIALIZE arr ay MAX PSE..27. 

100 DO 105 1=1,35 PRE 20 

105 HAX(I)=C . PRE 29 

C initialize RECORD COUNTER PRE 3j 

N1=0 _ _ . - - PRE 31 

C !IEA0 input data RECORO PRE 32 

110 REAOCINP.il) (IVAR(I) tl = li6L5j PRE 3 3 

11 F0RMAT(I3, ill, 12, 311, 412, 311, 013, 211, 513, 3512, 211) PRE 34 

IF(E0F(INP) )G0 TO l5u _ - PRE 35 

C—— INCREMENT RECORD COUNTER PRE 36 

Nl=Nl»l -PRE 37 

C STORE APPROPRIATE VARIABLES AND FIND'hAX VALUES PRE 38 



DO I2u 1=1., 35 ^ PR,E_JJJ5_. 

IS=Isi(I) PRE 4u 

IF(IVARCIS) .LE.HAX(I))G0 TO 120 _ . PRE 41 

MAX(I)=IVAR(IS) PRE 42 

120 KVAR(I)=IVAR(IS) _ PRE 43 

C WRITE OUTPUT RECORO IN BINARY PRE 44 



_MRlTE(IOUT)iKVAMI) tl?lid5)__ ^PS£_.4.5._ 

GO TO lib PRE 4b 

C MRITc FILE MARK, MAX. VALUEE _ PRE 47 

150 cNDFILE I OUT ' PRE 48 

WRITE (61, 12)LEVEL, Nl, (MAX(I) ,1=1,35) PRE 49 

12 F0RMAT(2I5,/, (7 #1014)) PRE 5j 

GO TO 90 _ .PR.E...1.1.._ 

C----’-'CONVERT LEVEL‘2 ANO FIND MA“x VALUES PRE 52 

2J0 DO 205 1=1,47 _ _ _ _ PRE 53 

235 MAX(I)*0 PRE 54 

N1=0 PRE 55 

210 REAO(INP,l3) (IVAR(I) ,1=1,135) PRE 56 

13 FORMAT (13. 1 2. 13 , 51 1 , 512 , Oil , 1012 ,2 41 1 , 12 , 121 1 , 12/ . fRE.__57 . 

15X3411,12,1513,612,211)' " ' “ ’ PRE 58 

iF(EDF(INP) )GO TO 25J _ PRE 59 

■4l=Nltl PRE 6. 
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0S3 PORTRAN VERSION 2.1 TABLEF 02/26/72 1718 

"-.50 IF(IOP1.GT.O)GO TO 190 - - - TBL121 

C-- — IF 1 DIM. COL. TABLE 60 TO 160 TBL122 

. IF(IVAR2.LE.O)60 TO 160 TBL123 

C WRITE ROM AND COL. IDENTIFIERS TB1.12L 

HRITE(46»13)IVAR1,IVAR2»IZ» (JJ»JJ=1»ICT) TBL125 

GO TO 170 TBL126 

C WRITE 1 DIM. IDENTIFIER ON OUTPUT TBL127 

160 WRITE(46,14)IVAR1 TBL12B 

C-‘— SET ROM, COU.NTER..TO Z.ERO, TEL129 

170 IsO TBL130 

C COMPUTE SUBSCRIPT FOR ROM TOTAL TBL131 

175 IP2=IS0(I»ICT,W,IRT1,ICT1) TBL132 

C IF ROM TOT. <* 0 60 TO 180 . TBL133 

IF(ITBL(IP2).LE.O)GO TO 18 TBL134 

^C.--— COMPUTE SUBSCRIPT FOR FIRST CELL IN LEVEL TBL135 

IPl3lS0(I,0fK,IRTl,ICTl) TBL136 

C WRITE ROM NUMBER, TABLE VALUES FOR ROM I TBL137 

MRITE(46,15)I, (ITBL( JJ) , JJ=IP1,IP7,IRT1) TBL138 

C INCREMENT ROM COUNTER . TBL139 

180 Isl^l TBL140 

C IF MORE ROMS. GO TO 175 I.BL141 

IF(I.LE.IRT)GO TO 175 TBL142 

CO TO 200 .. _ . .. .. TBL143, 

——SINGLE DIMENSION ROM DISPLAY TBL144 

COMPUTE SUBSCRIPT OF FIRST ARRAY LOCATION TBL145 

190 IP1:IS0(0,ICT,K.IRT1,ICT1) TBLT46 

j;--— -COMPUTE SUBSCRIPT OF LAST. ARRAJLL.OC.A_tI.ON .TBL147_: 

IP2=IS0(IRT,ICT,K,IRT1,ICT1) TBL148 

C COMPUTE NO. OF CATEGORIES . . TBL149 

IP*IP2-IP1+1 TBL150 

C DETERMINE LENGTH OF FIRST ROM _ TBL151 

IP=MIN0(IP,21) TBL152 

IPL = IP-1_ TBL.1,5.3 

C WRITE variable ID AND CAT. NO. FOR FIRST LINE TBL154 

MRITEC46,16)IVAR1,IZ,(JJ, JJ=1,IPL) _ .. . -TBL155 

C WRITE TABLE VALUES TBL156 

MRITE(46,19) (ITBL(JJ),JJ=IP1,IP) , TBL157 

C— .-COMPUTE FIRST SUBSCRIPT FOR SECOND LINE TBL158 

IP1=IP1+21 ^ TBL159... 

C— — IF START OF SECOND LINE > LAS'T ARRAY SUBSCRIPT 60 TO 100 TBL160 

195 IF(IP1.GT.IP2)G0 TO 100 TBL161 

C- COMPUTE START POINT OF SECOND LINE CAT, NO. ' TBL162 

IPS*IPL+1 TBL163 

C- COMPUTE NO. OF CAT. REMAINING TBL164 

IP»1P2-IP1*1 IBL165..^, 



IP*MIN0(IP,20) , TBL167 

C COMPUTE LAST VALUE FOR CATEGORY NUMBERS FOR THIS LINE TBL168 

IPLaIPS+IP-1 _ TBL169 

C— — COMPU’^E LAST SUBSCRIPT FO ARRAY IN THIS LINE TBL170 

IP=IP+IP1-1 TBL171 

C-— --WRITE CAT. ID NUMBERS tBL172 

MRITE(46,20M JJ, JJ=IPS,IPL) TBL173 

C—— WRITE TABLE VALUES FOR THIS LINE TBL174 

MRITE(46,21MITBL(JJ), JJ=IP1,1P) TBL175 

C— COMPUTE SUBSCRIPT FOR TABLE VALUES TO START NEXT LINE TBL176 

IPlsIPl+20 T8L177 

GO TO 195 ” TBL178 

C INCREMENT LEVEL COUNTER T8L179 

200 <=K+1 T8L180 

-i- 
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120 REAO(IOATA) (JVAR(J),J=1,NV) . 

C IF EOF GO TO 130 

IF(EOFdOATA) )GO TO 130 

C OETERHINE ROW SUBSCRIPT VALUE 

JV1=JVAR(IVAR1) 

JV2=JV3=0 

C IF AT least two DIM. DETERMINE COL. SUBSCRIPT 

IF(IVAR2.LE.O»GO TO 125 

JV2 = JVARdVAR2) ^ 

C IF 3 DIM. DETERMINE LEVEL SUBSCRIPT 

IF(IVAR3.LE.O)GO TO 125 

JV3=JVAR(IVAR3) 

C COMPUTE SINGLE DIMENSION SUBSCRIPTS 

C COMPUTE CELL SUBSCRIPT 

125 IP=IS0(JV1, JV2,JV3,IRT1, ICTl) 

C INCREMENT CELL FRFOUENCr 

ITBL(IP)=ITBL(IPI»1 

C IF 1 DIM. ,SET SUBSCRIPT TO VALUE FDR COL. TOT. CELL 

IFTICT.LE.O) JV1 = IRT _ 

C COMPUTE SUBSCRIPT FOR ROM TOTAL 

IP=IS0(JV1, ICT, JV3,IRTljt_ICU.) 

ITBL(IP) = ITBHIP) ♦! 

C- IF 1 DIM. GO TO 120 _ . 

IF(ICT.LE.0)GO TO 120 

C COMPUTE SUBSCRIPT FOR COL. TOTAL . _ . _ _ _ 

IP=IS0(IRT, JV2,JV3,iRTl, ICTl) 

tTBL(IP)=ITBHIP)*l __ 

C COMPUTE SUBSCRIPT FOR GRAND tdTAL(2 DIM.) OR LEVEL TOTAL (3 DIM.) 

IP=ISO(IRT, ICT, JV3,IRT1,ICT1) 

ITBL(IP)=ITBL(IP)*1 

IF 2 DIM. TABLE OR TOTAL LEVEL NOT OESIREO(IOP2<0) IN 3 DIM. 

GO TO 120 

_IF<ILT*IOP2.LE.O)GO TO 120 

0.---*-COMPUtE SUBSCRIPT FOR GRAND TOTAL OVER LEVELS 

IP=IS0(IRT, ICT, ILT,IRT1, ICTl) _ 

ITBL(IP)=ITBHIP)*1 

C SUBSCRIPT FOR CELL IN TOT. LEVEL 

IP=IS0(JV1, JV2,ILT,IRT1,ICT1) 

ITBHIP)_=ITBHIP)*1 

C— — -ROW TOT. IN TOT.' LEVEL 

IP=IS0(JV1, ICT, ILT,IRT1,ICT1) 

ITBL(IP)=ITBL (IP)*1 

C COL. TOT. IN TOT. LEVEL _ 

IP=IS0(IRT, JV2,ILT,IRT1,ICT1) 

_ ITBL(IP)=ITBL (IP) ♦! 

GO TO 120 

C PRINT TABLES SECTION 

C PRINT TABLE ID ARRAY 

130 CALL HEAOER(NTBL,LSORT,NDS,0) _ _ _ 

C SET LEVEL COUNTER TO ZERO 

_ K = 0 _ 

C-;.— IF not 3 DIM. TABLE GO TO 150 
IF(IVAR3.LE.O)GO TO 150 

C COMPUTE SUBSCRIPT FOR LEVEL GRAND TOTAL 

140 IPlsISOdRT, ICT, K,IRT1, ICTl) 

C IF TOTAL <= ZERO GO TO 200 

IF(ITBL(IP1).LE.O)GO TO 200 

C— level IOENTIFIERS'ON output LUN 
HRITE(46, 12)K,IVAR3 
C IF 1 DIM. ROM TABLE GO TO 190 



TBL 61 
TBL 62 

_TBL_ 63 
TBL 64 
TBL 65 
TBL 66 
TBL 67 
TBL 68 

_TBL_.6q 
TBL 70 
TBL 71 
TBL 72 
TBL 73 
TBL 74 

JBL 

TBL 76 
TBL 77 
TBL 7f. 
TBL 79 
TBL 60 

_TBL_81 
TBL 82 
TBL 83 
TBL 84 
TBL 85 
TBL 86 
TBL_87 

‘tBL 88 
TBL 89 
TBL 90 
TBL 91 
tBL 92 
TBL 93 
TBL 94 
TBL 95 
TBL 96 
TBL 97 
TBL 98 
T^L _99 

Tbl“i oo" 

TBLIOI 

T8L102 

TBL103 

TBL104 

TBL105 

'TBL106 

TBL107 

TBLlOa 

TBL109 

TBLllO 

TBLlll 

TBL112 

TBL113 

TBL114 

TBL115 

TBL116 

TBL117 

TBL118 

TBL119 

TBL120 
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SUBROUTINE TABLEF( IOATA,LSORT,LUNTBLiNTBL|NOS) 

c this routine takes the oata recoro output froh the sort routine 

— C AND PRODUCES CONTINGENCY. TABLES .0F_.0NEj_rH0i_0R_JHREE_01HENS.IQNS. 

C lOATAlLUN OF SCRATCH OATA RECORDS 

C LSORTILEVEL OF SORT . 

C LUNTBLILUN OF TABLE CONTROL INFO. 

C NTBLITABLE NUHBER COUNTER . .. . 

C NOSIOATA SET NUHBER 

_C-----ARRAt NNI NUMBER OF. VARIABLES . AT.EACH_LEVEU 

C ARRAY LIHILARGEST VALUE OF EVERY VARIABLE 

C ARRAY IQIOUHHY ARRAY 

COHHON/OATA/NN(7) f IQ ( 14) « ^IH ( 2341 

C ARRAY ITBLI ARRAY FOR STORAGE OF CONTINGENCY TABLE 

C ARRAY JVARIINPUT RECORO STORAGE 

^OIMENSION ITBL(25000), JVAR(99) 

C REHINO TABLE CONTROL LUN 

REWIND LUNTBL _ _ 

C— — READ VARIABLE ANO OPTION VALUES FOR TABLE 
100 REAO(LUNTBLflO)lVARlfIVAR2fIVAR3fIOPlf I0P2 
C-— -RETURN IF EOF 

IF(E.0FCLUNTBL)J RETURN 

C -SET COLUMN ANO ROW SIZE LIMITS ANO OUHMY VARIABLE TO ZERO 

ICT«ILT*IZaO 

C INCREMENT TABLE COUNTER 

NTBL=NTBL*1 

IF(IVAR1.GT.O)GO TO 105 

C PRINT ERROR MESSAGE ANO HEADER O N OUTPUT L UN 

call HEAOER(NTBLfLSORTfNOSfO) 

HRITE(46,11) 

GO TO 100 

C WRITE ERROR MESSAGE FOR TABLES DELETED OUE TO ARRAY 30UN0S ERROR 

C FOR ITBL 

t,03_W_RlTE (46,22)LSORT, IVARl, I VAR2, IVAR3, IZEROjNOS 

22 F0RMAT(71 TABLES OELETEOf LEVELFf I5f / » >0 VARIABLES' F,3I6,/f 

IFO IZERO =F,I10f/fF0 OATA SET NUMBER FfI6) 

CO TO 100 

C FINO MAX. CATEGORY VALUE FOR VAR. 1 CROWS) 

105 IRTsLiMITCLSORTf IVARD^l 

C-r.rt:rJF 2 DIM. TABLE FINO MAX. CAT. VALUE_FOR COL. 

IFCIVAR2.GT.0) ICT=LIMIT(LSORTf IVAR2) *1 

C IF 3 OIM. TABLE FINO MAX. CAT. VALUE FOR LEVELS _ 

IFCIVAR3.GT.0) ILT=LIMIT(LSORTf IVAR3) *1 

C—— COMPUTE NO. OF COL. INCLUOING ROW TOTAL COL. 

IRT1=IRT*1 

C—— C OMPUTE NO. OF ROWS INCLUDING CO L. TOTAL R OW 

ICT1=ICT*1 

C—— COMPUTE NO, OF LEVELS INCLUOING OVER-LEVELS TOTAL LEVEL 
ILT1=ILT*1 

C COMPUTE NUMBER OF CELLS NEEOEO IN TABLE ARRAY 

IZER0=IRT1*ICT1*ILT1 

C IF REQUEST EXCEEDS ARRAY BOUNOS G O TO 103 

IF(IZERO.GT.25000)GO TO 103 

C CLEAR TABLE ARRAY 

00 110 I=lfI7ERO 
110 ITBLCD^O 

C— — REWINO OATA RECORO SCRATCH LUN 

REHINO IDATA _ _ 

C DEFINE NUMBER OF VARIABLES AT THIS LEVEL OF SORT 

NV=NN(LSORT) 

C — REAO OATA RECORO 



5 

6 

7 
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TBL 34 
tbl 35 
TBL 36 
TBL 37 
TBL 38 
_TBL_39 
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C IF MORE LEVELS GO TO 140 . . . . TBL161 

IF(K,LE.ILT>&0 to 140 TBL162 

GO TO 100 . ^ .TBL183 

10 FORMATdOi:.) TBL164 

11 FORMAT(/l TABLES DELETED. ERROR IN CONTROL CAROS, TBL165 

1 ROW VARIABLE <= 0#) TBL166 

12 FORMAT(/0 LEVEL #,I4,# OF VARIABLE #I4/» TBL187 

13 FORMAT(/0 VARIABLE #,!•♦,# IS ROWS#,/,# VARIABLE #,I4,# IS TBL16« 

ICOLUMNS#,/, #0#, 13X2016,/, ( #. # , 16X , 19I6/.1 L TB1189 

14 FORMAT(#0 VARIABJ £ #,I4/) TBL190 

15 FORMAT(#0#,4X,21I6, ',(# #,16X, 1916/1) TBL191 

16 FORHAT(#0 VARIABi E # , 1 4 ,/, ( #0#, 4X , 2116) > TBL192 

19 FORMAT(# #,4X,21I6,/,(# # ,16X , 1916/) ) _ TBL193 

20 FORMAT(#0#, 10X,23I6) TBL194 

21 FORMAK# #,10X,?3I6) TBL195 

END TBL196 

NO ERRORS FOR TABLEF _ . 

P_ 62272 C .0 0 0 00 O_00_37_7 
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SUBROUTINE SORT « lOATA, INP,10UT , ISCR, NVAR, IVAR, LEVEL. LSORT, SRT 1 

INSORT.NREC.ISKP, ICRS, I0P4, NOS) SRT 2 

C-- — SUBROUTINE SORT SELECTS RECORDS ON THE BASIS , OF...USER..SUPPL lED SRT 3 

C CRITERION (REAL SORT) ANO/OR INFORMATION PROVIOEO FROM SORTS AT SRT 4 

C PREVIOUS LEVELS(N0MINAL SORT). ROUTINE WRITES COMPLETE DATA RECOR(3S SRT 5 

C ON LUN ISCR FOR USE BY TABLING ROUTINE ANO WRITES PARTIAL RECORDS SRT 6 

C ON LUN lOUT FOR USE IN SORTING SUBSEQUENT LEVELS SRT 7 

C IOATAILUN of data records to be sorted SRT 8 

C INPILUN OF scratch SORT ID RECORDS SRT_ 9 

C lOUTILUN C" SCRATCH SORT ID RECORDS FOR NEXT LEVEL SRT io 

C ISCRILUN OF SCRATCH DATA RECORDS FOR TABLING ROUTINE SRT 11 

C NVARINUMBER OF VAR. /LEVEL COMBINATIONS FOR REAL SORT SRT 12 

C IVARISTORAGE ARRAY FOR VARIABLE NUMBERS SRT 13 

C LEVEL (STORAGE ARRAY FOR WITHIN VARIABLE LEVEL NUMBERS SRT 14 

C .LSORT ILEVEL OF SORT SRT_15 

C MSORT ITABLING INDICATOR SRT 16 

C ISKPISKIP indicator =2 WHEN SORTING FROM LEVEL 3, =1 OTHERWISE SRT 17 

C ICRSIINOICATES ORDER OF CROSS SORT BETWEEN LEVELS 2 ANO 3 SRT 18 

C I0P4IIN0ICAT0R for means ANO PERCENTAGE ANALYSIS FOR LEVEL 4 SRT 19 

COMMON/OATA/NN(7) SRT 20 

Cr'.— NOISTORES id INFU. needed for sorting SRT_21 

C JVARI INPUT ARRAY FOR DATA RECORD SRT 22' 

C LVARISTORES OUTPUT NUMBERS OR PERSONNEL NUMBERS WHEN CROSS SRT 23 

C SORTING LEVELS 2 ANO 3 SRT 24 

DIMENSION IVAR(50) .LEVEL(50).NO(4) ,JVAR(99) .LVAR(220) SRT 25 

REWIND IL'ATASREWINO INPSREWINO ISCR2REWINOIOUT SRT 26 

C-rr--I.NITI ALITC STORAGE. AND FLAGS SRT_27 

IGRPsO ■■ SRT 28 

EOFMi^O. SRT 29 

DO 80 1=1,220 SRT 30 

SO LVAR(I)=0 SRT 31 

C INITIALIZE RECORD COUNTER, NO. OF VAR. READ FROM INP, ANO LARGEST SRT 32 

C IN0EX..NUM8ER OF VAR. USED IN_IO_CHFCK SRT__33 

NREC=LS1=LS11=0 SRT 34 

C WRITE SORT INFORMATION ON OUTPUT LUN SRT 35 

C IF NOMINAL SORT GO TO 70 SRT 36 

IF(NVAn.LE.0)GO TO 70 SRT 37 

HRITE(46,10)NO.',LSORT,ICRS,NVAR, (I VAR(J) .LEVEL (J) ,J=1, NVAR) SRT 38 

10 FORMAKxl DATA SET Y,I3,5X,X SORT LEVEL Y,I5,5X,# ICRS F,I5, _SRT 39 

15X,X NUMBER OF VA RIABL E/L E VELSF , 15 , / i FO VARIABLE LEVEL F,/, SRT'Vo ‘ 

250(17, 4X, 17/) ) . SRT 41 

GO TO 75 SRT 42 

70 WRITE(46,11)N0S, LSORT, ICRS SRT 43 

11 F0RMAT(F1 DATA SET F,.'3,5X,F SORT LEVEL Y,I5,5X,F ICRS F,I5,5X, SRT 44 

1///.F NOMINAL SORT _****_*±t) SRT_45 

C IF FIRST LEVEL GO TO 90 SRT 46 

75 IF(LSORT.LE.l)GO TO 90 SRT 47 

C DEFINE, NO. OF VAR. READ FPOM SORT LUN(LS1),AN0 LARGEST SRT 48 

C INDEX NO. OF VAR. USED IN 10 CHECK SRT 49 

LS1=1 SRT 50 

IF(ICRS.EQ.2) LS1 = 2 SRT 51 

L SI 1= ICRS " SRT 52 

IFdCRS.EQ. 0) LS11 = 1 SRT 53 

IF(LS0RT.LT.4)G0 TO 90 SRT 54 

LS1=2 SRT 55 

LS11=2 SRT 56 

IF(ISKP.EQ.2) LS11 = 3 _ _ SRT 57 

C-— -DEFINE NO. OF VAR. PER DATA RECORD ' ' " ' ' SRT 58 

90 NVsNN(LSORT) SRT 59 

C-— -initialize check values SRT 60 



ci V- 0 



331 



APPENDIX 6 (cont'd.) 



SORT- 2 



0S3 FORT-<AN VERSION 2.1 SORT OZ/Zi/TZ 171» 

<VAR = KVARl:NO(l)*0 ... SRT 61 

C ;?eAD data record SRT 62 

100 READ(IOATA) ( JVAR ( J) , J=1,NV) SRT 63 

IF(EOF(IOATA) )GO TO 200 ' SRT 6<* 

C IF LEVEL 1 REAL SORT GO TO 130 . SRT 65 

IF(LS1.LE.O.ANO.NVAR.GT.O)GO TO 130 SRT 66 

C IF LEVEL 1 N01INAL SORT GO TO 160 SRT 67 

IF(LSl.LE.0)GO TO 160 SRT 66 

C IF FIRST DATA RECORD GO TO . 180 SRT 69. 

IF(NO(1).LE.O»GO TO 180 SRT 70 

C INITIALIZE COUNTER SRT 71 

105 KK=0 SRT 72 

0 ID CHECKI COMPARE DATA RECORD ID TO SCRATCH INPUT RECORD ID SRT 73 

DO 110 I=1,LS11,ISKP SRT 7k 

KK=KK*1 SRT 75, 

IF( JVAR(I) .LT.NO(KK) )GO TO 100 SRT 76 

IF(JVARd) ,GT ,NO(KKnGO TO 180 SRT 77 

110 CONTINUE SRT 78 

0 IF nominal sort go to 160 SRT 79 

. IFINVAR.LE.OI GO TO 160 SRT 80 

130' /.VAR1 = NV1 = NV2 = C SRT 81 

0 -Variable sorti check data record against all var. /level combinationssrt 82 

DO 150 I=1,NVAR _ SRT 83 

<VAR=IVAR(I) SRT 84 

0 IF SAME VARIABLE AGAIN GO TO 140 _ SRT 65 

IF(KVAR.EQ.KVAR1)G0 to' 140 SRT 86 

0 IF record OOESNVT hatch go to 10 0 SRT 67 

IF(NV1.NE.NV2)G0 TO 100 SRT 88 

KVAR1=KVAR . SRT 89 

C INCREMENT VARIABLE COUNTER SRT 90 

NV1 = NV1*1 . ,. SRT 91 

C IF NO MATCH BETWEEN DATA VALUE ANO CRITERION VALUE GO TO 150 SRT 92 

140 IF(JVAR(KVAR) .NE»LEVEL(I> >G O TO 150 SRT. 93 

0 .-increment LEVEL COUNTER SRT 94 

NV2=NV2*1 _ _ . _ _ SRT 95 

150 CONTINUE SRT 96 

0 IF record DOESN/T match go to 100 . . SRT 97 

IF(NV1.NE.NV2)G0 TO 100 SRT 98 

C-> INCREMENT RECORD COUNT S RT 99 

160 NREC=NREC*1 ~ SRTIOO 

C IF CROSS SORT BETWEEN LEVELS 2 ANO 3 GO TO 300 _ SRTlOl 

IFdCRS.EQ.llGO TO 300 ' SRT102 

C IF SORT LEVEL <4 WRITE ID SORT INFO. ON SCRATCH OUTPUT LUN SRT103 

IF(LSORT.LT.4)WRITEIIOUT) IJVARU),J=l»LSORT) SRT104 

C IF TABLES TO BE PRODUCED WRITE FULL RECORD Or^ISCJ SRT105. 

IF (MSORT.GT.O) WRITE (ISCR) ( JVAR ( J) » Jsi» NV) SRT106 

C IF MEANS ANO PERCENTAGE ANALYSIS BUT NO TABLES WRITE FULL RECORD SRT107 

C ON LUN ISCR SRT108 

IF(IOP4.6T.O.ANO.MSORT.LE.O)WRITE(ISCR)«JVAR(J),J*1,NV) . SRT109 

GO TO 100 SRTllO 

0 i^eAD HEADER ID FROM SCRATCH INPUT LUN SRTlll 

180 REAO(INP) (NO(I) ,I=1,LS1) * ' SRT112 

IF(EOFdNP) )GO TO 200 _ _ SRT113 

GO TO 105 * SRT114 

0 IF EOF ANO NOT FIRST LEVEL OF CROSS SORT CO TO 205 SRT115 

200 IFdCRS.NE. DGO TO 205 SRT116 

C SET EOF FLAG _ SRT117 

EOFM=l. “ " ” ■ ■ ” SRT118 

GO TO 410 SRT119 

0— — -IF NO RECORDS MEET CRITERION RETURN TO MAIN PROGRAM SRT120 



APPENDIX 6 (cont'd.) 



SORT-3 



0S3 FORTRAN VERSION 2,1 SORT 02/26/72 1718 



205 IF(NREC.LE. 0) RETURN 

C WRITE FILE MARK ON TABLE SCRATCH LUN 

ENOFILE . ISCR... 

C RETURN IS SORT LEVEL >3 

IF(LSORT«GT.3)RETURN 

C WRITE FILE HARK ON HEAOER OUTPUT LUN 

ENOFILE lOUT 

REWIND INPSRENINO lOUTtREWlNO ISCR 

-SET-SKIP. FLAG 

ISKP=1 

IF(LSORT.EQ.3.ANO.ICRS.EQ.O)ISKP:2 

IF(LS0RT.EQ.3.AN0.ICRS.EQ.2)ISKP32 

RETURN 

C PREPARATION OF SCRATCH LUN DATA AT ICRS-1 

_C--~-IF_ FIRST PROJECT OR NEW PROJECT NUMBER GO T O 400 
300 IFdGRP.LT. JVARdMGO TO 400 

C IF SORT LEVEL 3 GO TO 320 

305 IF(LSORT.EQ.3)GO TO 320 

C FLAG OUTPUTS IN LVAR(SORT LEVELs2) 

00 310 1=31*45 

LFIJVAR(I) ,LE.0)GO TQ_J10 

JV«JVAR(I) 

LVAR(JV)=1 
310 CONTINUE 

C WRITE DATA RECORO ON ISCR IF TABLES TO BE MADE 

IF(HSORT.GT.O) WRITE (ISCR) ( JVAR (J) * Jsi^NV) 

60. TO >00 

C FLAG PERSONNEL NUMBERS IN LVAR (SORT LEVEL^S) 

320 00 330 1=8,11 

IF(JVAR(I).LE.O)GO TO 330 
JV=JVAR(I) 

LVAR(JV)=1 



SRT121 
SRT122 
3RT123 
SRT124 
SRT125 
SRT126 
SRT127 
SRT128 
SRT129 
SRT130 
SRT131 
SRT132 
SRT133 
SRT134 
SRT135 
SRT136 
SRT137 
SRT138 
SRT139 
SRT140 
SRTl' 1 
SRT142 
SRT143 
SRT144 
SRTl 45 
SRT146 
_SRT147 
^T148 
SRT149 
SRT150 
SRT151 
SRTl 62 



C 



C 

C 

-C. 

c 




c 



c 

c 



_310_CJ0NTIHUE SRTl 53 

WRITE OATA RECORO IF TABLES TO BE MAOE SRT154 

IF(MSORT,GT,0)WRITE(ISCR) (JVAR(J),J=1,NV) . __ , _ SRT155 

GO TO 100 SRT156 

RESET GROUP(PROJECT NO.) FLAG TO 10 NUMBER OF NEXT PROJECT AND WRITE SRT157 

SCRATCH OUTPUT RECOROS FOR THIS GROUP SRT158 

■"--r.IF.NOT FIRST RECORO GO TO 410 SR Tl 59 

400 IF(IGRP.GT.O)GO TO 410 SRViBO' 

SET GROUP FLAG TO FIRST PROJECT NUMBER . SRT161 

IGRP=JVAR(1) SRT162 

. . GO TO 305 _ _ _ _ SRT163 

-—-WRITE OUTPUT SCRATCH RECOROS SRT164 

410.00 420. 1=1, 220... SRT165 

-—-GO TO 420 IF LVAR <=0 SRT166 

IF(LVAR(I) .LE.OIGO TO 420 SRT167 

WRITE(IOUT) IGRP,I SRT168 

RESET NON ZERO VALUES TO ZERO SRT169 

LVAR(I)=0 SRT170 

420 CONTINUE SRT171 

IF EOF FLAG >0 GO TO 205 SRT172 

IF(EOFM,GT,0)GO TO 205 SRT173 

RESET GROUP FLAG TO NEXT PROJECT NUMBER SRTl 74 

IGRP=JVAR(1> SRT175 

GO TO 305 SRT176 

ENO . SRTl 77 



NO ERRORS FOR SORT 
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Computer Program Flow Charts 




PROGRAM RDDE7 



^ START ^ 



Initialize Values, 
Write Monitor 
Header 
42 - 48 
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RDDE7- 
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APPENDIX 7 (cont'd.) 



RDDE7-3 
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RDDE7-5 
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APPENDIX 7 (cont'd.) 



FUNCTION ISO 



^ START ^ 



> 


/ 


Compute 

Subscript 


, — 1 


i 



C 



RETURN 



) 
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FUNCTION LIMIT 
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SUBROUTINE HEADER 




F 




'ip 
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APPENDIX 7 vcont'd.) 



SUBROUTINE MEAN4 




F 
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PROGRAM PRECON 






351 



APPENDIX 7 (cont'd.) 



PRECON-2 




* . i • \ 



o 

ERIC 



352 




APPENDIX 7 (cont'd.) 



SUBROUTINE TABLEF 
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APPENDIX 4 (cont'd.) 
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Call 

Header to 
VPrint ID Array/ 




, ERIC 



vH . 4 



357 



k. 



API'ENMa 7 (conc'd.) 











o 4 O 



APPENDIX 7 (cont'd.) 



TABLEF-7 
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APPENDIX 7 (cont'd.) 



TABLE! 







^ 

Compute TAS 
for First 
Category in 
Second Line 




NO 



Lompute btart 
of Second Line, 
No. of Cat. Left, 
TAS and Cat. 
Value for Last Cat 



In Row 



162-171 



146-153 
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APPENDIX 8 



? 



File Documentation 



DATA FILES 

(for use with all computer-stored data) 



SECTION 


LEVEL 


1 


1 - A 


Project Descriptors - 


Complete Set 


2 


1 - B 


Project Descriptors - 


Sort Set 


3 


2 - A 


Personnel Descriptors 


- Complete Set 


4 


2 - B 


Personnel Descriptors 


- Sort Set 


5 


3 


Output Descriptors 




6 


4 


Scaled Data 




7 


5 


Standards Data 




8 


6 


Task Data 




9 


7 


Enabler (KSS) Data 





, 



O 

ERIC 



^7'// 375 



APPENDIX 8 (cont'd) 
Section 1 



LEVEL 1-A: PROJECT DESCRIPTORS 

(complete set of variables) 



Form 02 Data 



Variable 


Digits 


Source 


IZPe 


1. 


* Project Number 


3 


Front Page 


c 


2. 


* Project Focus (Set Y) 


1 


F.P.-A 


c 


3. 


* Project Complexity 


1 


F.P.-B 


c 


4. 


* Project Stage 


1 


F.P.-C 


c 


5. 


O.E. Region 


2 


F.P.-D 


c 


,7, 8. 


* Secondary Foci 


1 each 


F.P.-E,F 


,G C 


9. 


* Date of Data Collection 


2 


4 


Mo. 


10. 


* Date of Data Collection 


2 


4 


Yr. 


11. 


* Project Duration 


2 


7 


Mos . 


12. 


* Agency Type 


2 


8 


C 


13. 


* Total Project Funding 


1 


9a 


C 


14. 


* Current Fiscal Year Funding 


1 


9b 


c 


15. 


Last Fiscal Year Funding 


1 


9c 


r 


Fundin^i Source (12-19) : 








16. 


U.S.O.E. 


3 


lOa-i 


% 


17. 


Federal other than OE 


3 


lOb-j 


% 


18. 


State 


3 


lOr-b 


7. 


19. 


Private Foundation 


3 


lOd-1 


% 


20. 


Local School District 


3 


101-m 


% 


21. 


College or University 


3 


lOf-m 


% 


22. 


Industry 


3 


lOg-o 


ay 

/o 


23. 


Other 


3 


lOh-p 


ay 



APPENDIX 8 (contM) 



Variab le 



Digits 



Form 02 
Source 



Data 

IZE£ 



1’4. 


Primar>' Source of Funds 


1 


LI 


C 


25. 


* Number of Funding Sources 


1 


±2 


N 


26. 


Percent of Total Project Funding: 
Current Year 


3 


13a 


% 


27. 


Percent of Total Project Funding: 
Last Year 


3 


13b 


% 


2C. 


Percent of Total Project Funding: 
Next Year 


3 


13 c 


% 


Staf f 


Size (25-44) : 








29. 


* Professional Staff; Currently 


2 


14 a 


N 


30. 


* Professional Staff: Current FY 


2 


14f 


N 


31. 


Professional Staff: Last FY 


2 


14k 


N 


32. 


Professional Staff: Next FY 


2 


14p 


N 


33. 


* Clerical Support: Currently 


2 


14b 


N 


34. 


* Clerical Support: Current FY 


2 


14s 


N 


35. 


Clerical Support: Last FY 


2 


141 


N 


36. 


Clerical Support: Next FY 


2 


14q 


N 


37. 


* Technical Support: Currently 


2 


14c 


N 


38. 


* Technical Support: Current FY 


2 


14h 


N 


39. 


Technical Support: Last FY 


2 


14m 


N 


40. 


Technical Support: Next FY 


O 


14 r 


N 


41. 


* Subcontract Professionals : Current 


2 


14d 


N 


42. 


* Subcontract Professionals : Current FY 


2 


14i 


N 


43. 


Subcontract Professionals : Las t FY 


2 


14n 


N 


44. 


Subcontract Professionals: Next FY 


2 


14s 


N 
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APPENDIX 8 (cont'd) 



Variable 








Digits 


Form 02 
Source 


Data 

Type 


45. 


* Other 


Personnel : 


Currer.tly 


2 


14e 


N 


46 . 


* Other 


Personnel : 


Current FY 


2 


14j 


N 


47. 


Other 


Personnel : 


Last FY 


2 


lAo 


N 


48. 


Other 


Personnel : 


Next FY 


2 


14 1 


N 



Degree Levels of Staff (45-58) : 



49. 


ic 


Doctorate : Profes.slonal Staff 


2 


15a 


N 


50. 




Doctorate : Support Staff 


O 


15h 


N 


51. 


M 


Specialist /Professional : 
Professional Staf ^ 


2 


15b 


N 


52. 




Speciaiist/Frofessional : 
Support Staff 


2 


15i 


N 


53. 


* 


Masters: Professional Staff 


2 


15c 


N 


54. 




Mast ^rs: Support Staff 


2 


15j 


N 


55. 




Bachelors ; Professional Staff 


2 


I5d 


N 


56. 




Bachelors ; Support Staff 


2 


15k 


N 


57. 




Associate: Professional Sta.ft 


2 


15e 


N 


58. 




Associate: Support Staff 


2 


151 


N 


59. 


ic 


Technical License: Professional Staff 


2 


15f 


N 


60. 




Technical License: Support Staff 


2 


15m 


N 


61. 


ic 


Other Degree Level: Professional Staff 


2 


15g 


N 


62. 




Other Degree Level; Support Stcff 


2 


15n 


N 


63. 


ic 


Number of Reports and Documents 
Finished 


2 


16 


N 


64. 


ic 


Inter-Institutional Alignments 


.1 


17 


C 


65 


ic 


OutDut Commitments 


1 


18 


C 



KEY TO DATA TYPES: 

C = coded 

0 = occurrence of them 
N ® number ount 

* Variable included in Level 1-B, for sortingi^'^ata in Levels 3, 4, 5, 6. 



APPENDIX 8 (cont'd) 
Section 2 



LEVEL 1-B: PROJECT DESCRIPTORS FOR SORTING 

(for sorting Levels 2, 3, 4, 5, 6, or 7) 



Variable 


Digits 


Form 02 
Source 


Data 

Jypg 


1. 


Project Number 


3 


Front Page 


C 


2. 


Project Focus (Set Y) 


1 


F.P.-A 


C 


3. 


Project Complexity 


1 


f.p.-b 




4. 


Project Stage 


1 


F . P . -C 


c 


6, 7. 


Secondary Foci 


leach 


F.P.E,F,G, 


c 


8. 


Date of Data Collection 


2 


4 


Mo. 


9. 


Date of Data Collection 


2 


4 


Yr. 


10. 


Project Duration 


2 


7 


Mos . 


11. 


Agency Type 


2 


8 


c 


12. 


Total Project Funding 


1 


9a 


c 


13. 


Current Fiscal Year Funding 


1 


9b 


c 


14. 


Primary Source of Funds 


1 


11 


c 


15. 


Number of Funding Sources 


1 


12 


N 


16. 


Professional Staff : Currently 


2 


14a 


N 


17. 


Professional Staff: Current FY 


2 


14f 


N 


18. 


Clerical Support: Currently 


2 


14b 


N 


19. 


Clerical Support: Current FY 


2 


14g 


N 


20. 


Technical Suppor t : Currently 


2 


14c 


N 


21. 


Technical Support: Current FY 


2 


14h 


N 


22. 


Subcontract Professionals : Currently 


2 


I4d 


N 


23. 


Subcontract Professionals: Current FY 


2 


14i 


N 


24. 


Other Personnel : Currently 


2 


14e 


N 


25. 


Other Personnel; Current FY 


2 




N 



O 

ERIC 



381 , 

' 6 4 



Ai^t'tNUiA Uonc d; 



'ari ab Ic 




I) 1 f>l t.s 


I'Drm 02 
.Sou rce 


l,);it ,1 


26. 


Doctorate : Professional Staff 


2 


I5a 


N 


27. 


Special is t/Prof essional : 
Professional Staff 


2 


15b 


N 


28. 


Masters : Professional Staff 


2 


15c 


N 


29. 


Bachelors: Professional Staff 


2 


I5d 


N 


30. 


Associate; Professional Staff 


2 


15e 


N 


31. 


Technical License: Professional Staff 


2 


I5f 


N 


32. 


Other Degree Level: Professional Staff 


2 


15g 


N 


33. 


Number of Reports and Documents Finished 


2 


16 


N 


34. 


Inter-Institutional Alignments 


1 


17 


C 


35. 


Output Commitments 


1 


18 


C 



KEY TO DATA TYPES: 

C = coded 
0 = occurrence 
N = number count 



APPENDIX 8 (cont'd) 



Section 3 

LEVEL 2-A: PERSONNEL DESCRIPTORS 

(complete set of personnel variables) 






O 

ERIC 









Form 03 


Data 


Variable 


Digits 


Source 


Tyj>e 


1. 


* Project Number 


3 


top 


c 


2. 


* Interviewee Number 


2 


top 


c 


3. 


* FTE of Interviewee on Project 


3 


top 


N 


4. 


* Manager vs Staff /Line (and level) 


1 


3/ 


c 


5. 


* Sex 


1 


5. 


c 


6. 


* Age 


1 


6. 


c 


7. 


* Highest Degree Held 


1 


7. 


c 


8. 


* Number of Professional Pubs 


1 


9. 


c 



9. 


* Major For Highest 


Degree 


(1) 


2 


10. 


c 


10. 


* Major For Highest 


Degree 


(2) 


2 


10. 


c 


11. 


Minor For Highest 


Degree 


(1) 


2 


11. 


c 


12. 


Minor For Highest 


Degree 


(2) 


2 


11. 


c 


13. 


Minor For Highest 


Degree 


(3) 


2 


11. 


c 


14. 


* Annual Salary 






1 


12. 


c 


Professional Memberships ' 


(15-21) : 










15. 


AERA Member 






1 


13a 


0 


16. 


NEA Member 






1 


13b 


0 


17. 


AVA Member 






1 


13c 


0 


18. 


APGA Member 






1 


13d 


0 


19. 


APA Member 






1 


13e 


0 


20. 


AECT Member 






1 


13f 


0 


21. 


Other Membership 






1 


13g 


N 
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API'F.NIMX 8 (cont’d) 



Form 03 Hat.i 



'ariable 


Digits 


Source 




Prior 


Work Experience (22-30)! 








22. 


* College Teaching Experience 


2 


14a 


yrs . 


23. 


* College Research Experience 


2 


14b 


yrs . 


24. 


* Public School Experience 


2 


14c 


yrs . 


25. 


* State or Nat’l Educ. Agency Experience 


2 


14d 


yrs . 


26. 


* Educ. R & D Center Experience 


2 


14e 


yrs • 


27. 


* Present Organization Experience 


2 


14f 


yrs • 


28. 


* Other Educ. or Research Experience 


2 


14g 


yrs • 


29. 


* Total Educ. R.D.D.E. Experience 


2 


16 


yrs . 


30. 


* R.D.D.E. Directing Experience 


2 


17 


yrs , 


31. 


* No. of Times Principal Investigator 


2 


18 


N 


32. 


* No. of Proposals Submitted 


2 


19 


N 


33. 


* No. of RDDE Projects Worked On 


2 


20 


N 


34. 


* No. of RDDE Projects Now On 


2 


21 


N 


Required Experience (35-37); 








35. 


RDDE Experience Required 


2 


26 


yrs . 


36. 


Administration Experience Required 


2 


27 


yrs . 


37. 


Management Experience Required 


2 


28 


yrs . 


38. 


Level of Academic Training Required 


2 


29 


C 


39. 


In-Service Training Received 


2 


30 


0 


Support Services (40-64); 








40. 


Equipment Cons truction 


1 


31a , 


0 


41. 


Printing 


1 


31b 


0 


42. 


Other Reproduction Services 


1 


31c 


0 
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APPENDIX 8 (cont'd) 



Variable 




Digits 


Form 03 
Source 


Data 

Type 


43. 


Photography 


1 


31d 


0 


44. 


Art Work and Illustrations 


1 


3le 


0 


45. 


Draf ting 


1 


3lf 


0 


46. 


Technical Writing 


1 


31g 


0 


47. 


Ed i ting 


1 


3lh 


0 


48. 


Secretarial Service, Other Than Typing 


1 


3li 


0 


49. 


Typing 


1 


31j 


0 


50. 


Purchase of Supplies and Equipment 


1 


31k 


0 


51. 


Library Holdings 


1 


311 


0 


52. 


Subscriptions to Journals /Periodicals 


1 


31m 


0 


53. 


Requests for Doc-.uments or Publications 


1 


3ln 


0 


54. 


Computer Analysis Services 


1 


31o 


0 


55. 


Computer Program VJriting 


1 


31p 


0 


56. 


S tatis tical Consul tat ion 


1 


31q 


0 


57. 


Audio-Visual Aids and Devices 


1 


31r 


0 


58. 


Subjects for Experimentation 


1 


31s 


0 


59. 


Travel Arrangements 


1 


31 1 


0 


60. 


Budgetary and Other Fiscal Accounting 


1 


3lu 


0 


61. 


Scoring of Test Items 


1 


3lv 


0 


62. 


Television Facilities and Equipment 


1 


3lw 


0 


63. 


No. of Other Support Services Cited 


1 


31x,y ,z 


N 


64. * 


Total Number of Support Services 


2 


31 


N 
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APPENDIX 8 (cont'd) 



Variable 




Digi ts 


Form 03 
Source 


Data 

Type 


Support 


Equipment (65-77): 








65. 


Dictating Equipment 


1 


32a 


0 


66. 


Desk Calculators 


1 


32b 


0 


67. 


Desk- top Compu ter 


1 


32c 


0 


68. 


Remote Computer Terminal 


1 


32d 


0 


69. 


On-site Computer 


1 


32e 


0 


70. 


Key-punch Machine 


1 


32f 


0 


71. 


Data Card Sorter 


1 


32g 


0 


72. 


Photographic Equipment 


1 


32h 


0 


73. 


Video Tape 


1 


321 


0 


74. 


Television Camera 


1 


32j 


0 


75. 


Readers for Microfiche or Microfilm 


1 


32k 


0 . 


76. 


No. of Other Support Equipment Cited 


1 


321,ra,n 


N 


77. * 


Total Number of Support Equipments 


2 


32 


N 


78. 


Trips Per Year 


1 


33a 


c 


79. 


Trip Duration 


1 


33b 


c 


80. 


Adequacy of Manpov/er 


1 


34 


c 


81. 


Adequacy of Time 


1 


35 


c 


82. 


Adequacy of Finances 


1 


36 


c 


Project 


Time Available (83-90): 








83. 


Professional Papers 


1 


37a 


0 


84. 


Speeches 


1 


37b 


0 


85. 


Conventions 


1 


37c 


0 


86. 


Current Literature 


1 


37d 


0 
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APPENDIX 8 (cont’d) 









Form 03 


Data 


Variable 




Digits 


Source 


IZ£e 


87. 


Proj ect Li terature 


1 


37e 


0 


88. 


Preparation of Proposals 


1 


37f 


0 


89. 


Contacts With Other Agencies 


1 


37g 


0 


90. 


Personnel Interaction 


1 


37h 


0 


Expected Activities (91-98): 








91. 


Preparation of Professional Papers 


1 


38a 


0 


92. 


Presentation of Speeches 


1 


38b 


0 


93. 


Conventions & Professional Meetings 


1 


38c 


0 


94. 


Reading of Current Professional Literature 1 


38d 


0 


95. 


Reading of Project Literature 


1 


38e 


0 


96. 


Preparation of Proposals 


1 


38f 


0 


97. 


Contacts With Other Agencies 


1 


38g 


0 


98. 


Personnel Interaction 


1 


38h 


0 


Guidance Provided (99-108): 








99. 


Project Time Lines 


1 


39a 


0 


100. 


Forecasting, Monitoring, and Accounting 


1 


39b 


0 


101. 


Specific Work Assignments (Daily /Weekly) 


1 


39c 


0 


102. 


General Statements of Functional Goals 


1 


39d 


0 


103. 


Occasional Reviews of Efforts and 










Remaining Work Goals 


1 


39 e 


0 


104. 


Frequent Review of Work Achievement 


1 


39f 


0 


105. 


Regular Staff Meetings for Project Review 


1 


39g 


0 


106. 


Individual Performance Review (Annual) 


1 


39h 


0 


107. 


Close Dally Working Contact 


1 


39i 


0 


108. 


No. of Other Guidance Means Cited 


1 


39j 


N 
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APPENDIX 8 (cont’d) 









Form 03 


Data 


Variable 


Digits 


Source 


Jy££ 


109. 


* Project Structure 


1 


40 


c 


110. 


* Internal Coordination Required 


1 


41 


s 


111. 


* External Coordination Required 


1 


42 


s 


112. 


* Number of Outputs Interviewed 


2 


top 


N 


113-127. 


* Output Number (s) — up to 5 Outputs 


3 each 


top 


c 


Persons Directly Supervised (114-117): 








128. 


Total Currently 


2 


44a 


N 


129. 


Project Currently 


2 


44b 


N 


130. 


Total Normally 


2 


44c 


N 


131. 


Project Normally 


2 


44d 


N 


Persons Under General Supervision (118-119): 








132. 


Total Currently 


2 


45a 


N 


133. 


Project Currently 


2 


45b 


N 


134. 


* Relationship to Project 


1 


46 


c 


135. 


* Project Role (Set X) 


1 


47 


c 




KEY TO DATA TYPES: 

C = coded 

0 = occurrence of item 
N = number count 
S = scaled 



^Variable included in Level II-B, Sort Set. 
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APPENDIX 8 (cont’d) 



Section 4 



LEVEL 2-B: PERSONNEL DESCRIPTORS 

(for sorting levels 3, 4, 5, 6, 7) 



Form 03 Data 



triable 


Digi ts 


Source 




!• 


Project Nunber 


3 


top 


C 


2. 


Interviewee Number 


3 


top 


C 


3. 


FTE of Interviewee on Project 


3 


top 


N 


4. 


Manager vs Staff/Line (and level) 


1 


3,4 


C 


5. 


Sex 


1 


5. 


C 


6. 


Age 


1 


6. 


c 


7. 


Highest Degree Held 


1 


7. 


c 


8. 


Number of Professional Pubs 


1 


9. 


c 


9. 


Major For Highest Degree (1) 


2 


10. 


c 


in. 


Major For Highest Degree (2) 


n 


10. 


c 


11. 


Annual Salary 


1 


12 


c 


Prior 


Work Experience (12-20) • 








12. 


College Teaching Experience 


2 


I4a 


yrs . 


13. 


College Research Experience 


2 


14b 


yrs . 


14. 


Public School Experience 


2 


I4c 


yrs . 


15. 


State or Nat'l Educ, Agency Experience 


2 


I4d 


yrs . 


16. 


Educ. R&D Center Experience 


2 


I4e 


yrs . 


17. 


Present Organization Experience 


2 


I4f 


yrs . 


18. 


Other Educ. or Research Experience 


2 


I4g 


yrs . 


19. 


Total Educ. RDDE Experience 


2 


16 


yrs . 


20. 


RDDE Directing Experience 


2 


17 


yrs . 
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APPENDIX 8 (cont'd) 



Variable 




Digi ts 


Form 03 
.Source 


Data 

Type 


21. 


No. of Times Principal Investigator 


n 


18 


N 


22. 


No. of Proposals Submitted 


2 


19 


N 


23. 


No. of RDDE Projects Worked On 


2 


20 


N 


24. 


No. of RDDE Projects Now On 


2 


21 


N 


25. 


Total Number of Support Services 


2 


31 


N 


26. 


Total Number of Support Equipments 


2 


32 


N 


27. 


Project Structure 


1 


40 


C 


28. 


Internal Coordination Required 


1 


41 


s 


29. 


External Coordination Required 


1 


42 


s 


30. 


Number of Products Interviewed 


2 


top 


N 


31-45. 


Output Number (s) — up to 5 Outputs 


3 each 


top 


c 


46. 


Relationship to Project 


1 


46 


c 


47. 


Project Role (Set X) 


1 


47 


c 



KEY TO DATA TYPES: 

C = coded 

0 = occurrence of item 
N = number count 
S = scaled 
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APPENDIX 8 (cont'd) 



i 
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t- 



Sec ‘.ion 5 

LEVEL 3: OUTPUT DESCRIPTORS 

(for sorting Le''els 2, 4, 5, 6, 7) 



Variable 




Digits 


Form 05- 
Source 


B Data 
Type 


1. 


Project Number 


3 


top 


c 


2. 


Output Identification Number 


3 


1st 


c 


3. 


Output Frequency 


3 


2nd 


N 


4. 


Output Completion Stage (Set W) 


1 


3rd 


c 


5. 


Structure of Output (Set A) 


1 


4th 


c 


6. 


Output Level (Set B) 


1 


5 th 


c 


7. 


Number of Data Respondents 


1 


6-9 


N 


8-11. 


Interviewee Number(s) — up to 4 persons 


3 each 


10th 


c 


12. 


Cluster of Outputs (Set DE) 


2 


10th 


c 


13. 


Primary Category of Output (Set FGHI) 


4 


11th 


c 


14. 


Project Focus (Set Y) 


1 


12 th 


c 


15. 


Character of Focal Output (Set Z) 


1 


13th 


c 


16. 


Output Character (Set C) 


1 


13 th 


c 


17. 


Output Function (Set a) 


1 


15-16 


N 


Varir oles 
4 , 5 and 


1, 2, 5, 6, 7, 8-11, 12, 13 and 16 are 

6. 


duplicated 


in Data 


Files 
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APPENDIX 8 (cont'd) 
Soc'^ion fi 



LEVEL 4; SCALED DATA 



Variable 






Digits 


Form 04 
Source 


Data 


1. 


Project Number 




3 


top 


C 


2. 


Interviewee Number 




*5 


top 


c 


3. 


Item Set 




n 


top 


c 


4. 


Scale Used 




2 


top 


c 


5-15. 


Item Set 1, Significance 
Activity , 11 Items) 


(General 


1 each 


— 


s 


16-77. 


Item Set 2, Significance 
Activity, 62 Items) 


(Specific 


1 each 


— 


s 


78-98, 


Addendum 




1 each 


— 


s 


99, 


No, of Specific Items Added 
(Item Set 2) 


2 




V 

L\ 
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APPENDIX 8 (cont'd) 



' 



I 



Section 7 



LEVEL 5: 



STANDARDS DATA 



Variable 



1 . 



2 . 



3. 



4. 

5. 

6 . 

7. 

8 . 



9. 



10 . 

11 . 

12 . 



Project Number 

Interviewee Number 

Output Identification Number 

Page Number (in Project Set Form 06 
Recap of Standards) 

Structure of Outputs (Set A) 

Level of Outputs (Set B) 

Character of Outputs (Set C) 

Clusters of Outputs (Set DE) 

Primary Categories of Outputs (Set FGHI) 

Structure of Standards (Set J) 

Self/Other Report (Set K) 

Primary Categories of Standards (Set LM) 



Form 06 Data 

Plaits Source Type 

3 top-lst C 

3 top-2nd C 

3 top-3rd C 

2 top-4 th C 

1 top-A C 

1 top-B c 

1 top-C C 

2 top-DE C 

4 top-FGHI C 

1 J C 

1 K c 

2 LM ■ C 
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APPENDIX 8 (cont'd) 



Section 8 

LEVEL 6 : TASK DATA 



Variable 




Digits 


Form 07 
Source 


Data 

Type 


1. 


Project Number 


3 


top-lst 


c 


2. 


Interviewee Number 


3 


top-2nd 


c 


3. 


Output Identification Number 


3 


top-3rd 


c 


4. 


Page Number (in Project Set of Form 07 
Recap of Tasks) 


2 


top-4th 


c 


5, 


Structure of Outputs (Set A) 


]. 


top-A 


c 


6. 


Level of Outputs (Se.t B) 


1 


top-B 


c 


7. 


Character of Outputs (Set C) 


1 


top-C 


c 


8. 


Clusters of Outputs (Se.. DE) 


2 


top-DE 


c 


9. 


Primary Categories of Outputs (Set FGHI) 


4 


top-FGHI 


c 


10. 


Clusters of Tasks (Set NO) 


2 


NO 


c 


11. 


Self/Other Report (Set P) 


1 


P 


c 


12. 


Primary Categories of Tasks (Set QR) 


2 


QR 


c 
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APPENDIX 8 (cont'd) 



i. 



Section 9 

LEVEL 7: ENABLER (KSS) DATA 

Variable 

1. Project Number 

2. Interviewee Number 

3. Output -Identification Number 

4. Page Number (in Project Set of Form 08 
Recap of Enablers) 

5. Structure of Outputs (Set A) 

6. Level of Outputs (Set B) 

7 . Character of Outputs (Set C) 

8. Clusters of Outputs (Set DE) 

9. Primary Categories of Outputs (Set FGHI) 

10. Structure of Enablers ;K, S, or S) (Set S) 

11. Self/Other Report (Set T) 

12. Primary Categories of Enablers (Set UV) 



Form 08 Data 
Digits Source Type 



3 

3 

3 
2 

1 

1 

1 

2 

4 
1 
1 
2 



top-lst C 
top-2nd C 
top-3rd C 
top-Ath C 



top-A 

top-B 

top-C 

top-DE 



S 

T 

UV 



C 

c 

c 

c 



top-FGHI C 



C 

C 

C 
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APPENDIX 9 
Profile Components 



Profile Content Attaf'hments 

A1 Title Page 

A2 Table of Contents 

A3 Synopsis 

A4 Contextual Map 

AS Organizational Structure Chart 

A6 Project Roster 

A7 Output Index 

A8 Table of Output Frequency of Occurrence by Category 

A9 Output Map 

Supplementary Attachments 

AlO Distribution of Responsibilities for Profile Items 
All Profile Transmittal Sheet (Team, Worksheet 20) 

A12 Profile Transmittal Sheet (Coordinators, Worksheet 21) 

A13 Profile Transmittal Sheet (Editor, Worksheet 22) 

A14 Profile Transmittal Sheet (Project Director, Worksheet 23) 
A15 Profile Rating Sheet (Attached to W23) 

A16 Profile Release Sheet (Attached to W23) 
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APPENDIX 9 



Profile Components 



Attachment 1 : Title Page 



A CASE PROFILE OF 

A PROJECT TITLED: Study and Development of Automated Ins tructional- 

Materials-Handling Program 



(AIMS Project) 

AN EDUCATIONAL DEVELOPMENT PROJECT CONCERNED WITH: Developing a compre- 

hensive automated system for handling information about instructional 
materials (book and nonbook) that meets the Los Angeles Unified School 
District needs and also serves as both a model and prototype system for 
large school districts throughout the country. 

A PROJECT OF: Los Angeles Unified School District 

A50 Narth Grand Street 
Los Angeles, California 90012 



Case Profile Number 15 
December 1970 



This case profile is one of twenty being prepared within a project grant 
for "The Generation of Information to Support Long Term Manpower Studies 
of and Planning for Training Programs in Educational Research, Develop- 
ment, Diffusion and Evaluation" from the National Center for Educational 
Research and Development, Office of Education, U.S. Department of Health, 
Education and Welfare. 



Prepared by 
Teaching Research 

A Division of the Oregon State System of Higher Education 
Monmouth, Oregon 97361 



APPENDIX 9 (cont’d.) 



Attachment 2 TABLE OF CONTENTS 

CHAPTER I: OVERVIEW 

Synopsis of the Project 

Objectives, Rationale, and Significance of the Project . . . . 

Context in Which the Project Operates 

Relationship to ocher agencies 

Time lines . 

Physical/environmental setting 

CHAPTER II; PARAMETERS OF THE PROJECT 

Project Structure 

Staff structure 

Project roster 

Outputs Generated 

Index of outputs 

Output map 

CHAPTER III: SUMMARY OF THE DATA 

Standards Held for Outputs 

Tasks Pertaining to Output Attainment 

Enablers Pertaining to Output Attainment 

f Knowledges 

Skills 

Sensitivities 

Interrelationships among the enablers 

Discussion of the Output Data 

CHAPTER IV: SUPPLEMENTARY DATA 

Classifications of Output Characteristics 

Summary of Staff Backgrounds 

Summary of Position Requirements and Support Systems 

SCI 
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9 

13 

15 

16 

16 

16 

16 

21 

21 

21 

21 

25 

25 

25 

27 



APPENDIX 9 (cont'd.) 







A2-2 



Significance of Various Categories of Work 27 

Sununnry of Project Funding 28 

Discussion of the Supplementary Data 29 

CHAPTER V; PROJECT DYNAMICS 31 

Operation of the Project Staffing Structure 31 

Tri-University structure 31 

Relationships among the directors 31 

The consultants 32 

The Issues Raised 32 

Content vs. Technological Foci 33 

Influence of Project Dynamics on Evolution of the Catalog ... 33 

CHAPTER VI: IMPLICATIONS FOR TRAINING 37 

APPENDIX: Listing of Output Standards, Tasks, and Enablers .... 39 



FIGURES 

1. Contextual map 4 

2. Project time line chart for major activities 5 

3. Project organizational structure 7 

4. Output map 14 



TABLES 



1. Project Roster of BOE Staff by Job Titles 8 

2. Output Standards Cited for Each Output Analyzed 17 

3. Process Standards Cited for Each Output Analyzed 18 

4. Tasks Cited for Each Output Analyzed. . 19 

5. Enabling Knowledges Cited for Each Output Analyzed 20 

6 Enabling Skills Cited for Each Output Analyzed 22 

7. Enabling Sensitivities Cited for Each Output Analyzed ... 23 

8. Classifications of Outputs Analyzed 26 

9. General Activity Significance 28 
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Attachtaent 3 ; Synopsis 



APPENDIX 9 (cont’d.) 



Synopsis of the Project (on 1 page) 
Title: 



Responsible Institution: 

Subcontractor(s) : (if any) 

Funding Source(s): 1. 

2 . 

Funding Duration: start and end dates. (x months) 

Observation Date: Month and year 

Present Stage of Development : . 

RDD or E Focus of Project: Educational 

Expected Outcome(s): 



Level of Funding and Duration: (level x of 7 levels) 

Agency Setting: 



Staff Summary (Current): 



Professional Support 



Total Full Time Equivalency (in 
man years) : 

Number of Personnel Assigned: 
Prime Contractor 
Subcontractor and Others: 



X 

X 



X 

X 



Professional Specialities of Staff (interviewees only) : 
(3). (2), , 
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APPENDIX 0 (cont'd) 



Attachment 4 ; Contextual Map 




A final version of a contextual map. 



I 



409 




APPENDIX 9 (cont'd) 
Attachment 5 : Organizational Structure Chart 




A project organlzaclonal structure. 




1' ■ I 411 



365 



APPENDIX 9 (cont’d) 



Attachment 6 ; Project Roster 



Project roster . The following staff members were interviewed dur- 
ing the onsite visitation of the interview team. 

Project Director . Assigned .25 full time equivalency (FTE) to this 
project, he worked on the proposal and was reassigned to the project 
for its duration when it was funded. He was also director of the 
earlier, related project of which REACT is a logical continuation. 
His responsibilities are administrative and supervisor, and he is 
primarily responsible for the final reporting from this project. 

School Administrator Training Package Coordinator . Hired during 
previous phase of the project, he was reassigned (1.0 FTE) to 
the project when the current contract was funded. He is responsi- 
ble for the completion of the Administrator Package, the MIDAS 
system, and shares responsibility for the testing design for the 
project. 

Teacher Training Package Coordinator . She is supervisor of the 
Student Training Package and Clearinghouse efforts, having been 
hired during a previous phase of the project and assigned to the 
current project from beginning date at 1.0 FTE. She is responsi- 
ble for the completion of the Teacher Training Package, and shares 
responsibility for the testing design for the project. 

Curriculum Writer . She was hired (1.0 FTE) for writing student 
training units at the project's inception. She worked for a short 
term on a previous project in a similar capacity. 

Programmed Instruction Specialist . Employed during a previous 
phase of the project, he rejoined the current effort at the be- 
ginning date with 1.0 FTE. He is responsible for development of 
the Writer Training process, consisting of interaction between 
curriculum writers and specialists in instructional technology. 

The following staff member (not interviewed) completes the list of pro- 
ject personnel. 

Computer Programmer . He is employed for the duration of the pro- 
ject at 1.0 FTE. 
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APPENDIX 9 (corn’d) 



Attachment 7 : Output Index 

P-01 . Computer-use Instructional Unit for Administrators . 

A written instructional plan for use in teaching 
school administrators to operate computers as a 
tool to enhance performance of their various adminis- 
trative duties. 

P-02. Computer-use Instructional Unit for Teachers A written 
Instructional plan for use in teaching teachers to oper- 
ate computers as a tool to extend their classroom teach- 
ing effort. 

P-03. Computer-use Instructional Unit for Students . A written 
instructional plan for use in teaching students to oper- 
ate computers as a learning tool. 

E-04. Instructional Unit-writer Training . A series of inhouse 
training sessions planned to increase the capability of 
the writers in writing instructional units. 

P-05. Computer Simulation of School Administration (MIDAS) A 
minature demonstration computer system of hypothetical 
school administration. This system includes pupil, per- 
sonnel, financial, facility, and curriculum data files. 

P-06. Field Testing Design . The written plan for evaluating 
the various instructional units being produced by the 
REACT Project. 

P-07. List of Field Test Sites . A refined list of locations 
which would cooperate in the conducting of field tests 
for the various REACT outputs. 

p-08. Clearinghouse Operational Plans . A written operational 
plan for the obtaining, classifying, and distributing 
of computer applications in school situations. 

P-09. Three Student Instructional Units (Application Packages) . 

Three written instructiona" plans to assist students in 
learning subject matter by employing the computer as an 
aid or tool. 



AI'l’l'NJlX 9 (cont'ci) 



Attachment 8: Table of Output frequency of Occurrence by Catcj’.ory 
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froJccC Output* 

Mo. 

f~01 CaColoR of ftrCorMn;* ObJ*c- 

Civ** In Cniliah, Cr*dc* 9-12 
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D*ry C*c*lo| 
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to Cetelog 

P-14 Journal Report (Project Officer) 

P-15 Joumel Report of Project Progree* 
(Project Director) 

P-16 Dleeeotlog Poeltlon Peper (Journal 
Article*) 

•P-17 Reeponee to Dleeeotlog Poeltlon Peper 
(Journal Article*} 
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P-19 Agende for inltlel Confereoce 
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P-23 Initial Field Tear Plea 

P-24 Llet of Field Teat Sites 

•P-25 List of Cooaultente (Coofiread) 

P-26 Propose 1 

•1-27 Coofere-ce, f* Id Repraaenteeiva* 

•1-21 Inltlel Work Conference (October,#!) 

1-29 2nd Work Conference (March, #2) 
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Claaalficatlcn Praquenclaa 



Output Chrtfafterl Ic" 


St ructura 


Tunct l.'n 


Ijjwl 


j rharactrr 

! (ProJucta onlv) 


Torplrtlon St.-we 


pec 


re n P 


1 1 C t : 


jTk t .11 12 


r : 3 4 5 6 


X 


I 


X 


X 


X 


X 


I 


X 


X 


X 


I 


I 


X 


X 


X 


I 


I 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


I 


X 


X 


X 


X 




X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


I 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 




' X 


X 


X 


X 




X 


X 


X 


X 




X 


X 


X 


X 




I 


0 

I ^ 


1 19 10 


3 S 22 


0 16 4 6 


19 4 3 3 1 0 



• Tha opeciflc output charactarlatlca art Idantlflad at follouej 



ftryclura 



functi on 



f - product 

a avant 

c - COAdltioB 



ps - aaccing 

n nan A*«anc 

f - production 



f| - focal 
c - coapontnc 
§2 - faclllcatlag 






Cbaractar 



Conplatlon Stagt 



k - kfiA#ladga 

C • Cachnolotr 
- InpUoentac Im 
I 2 - inforevitlon 




1 - conplatad ovar ont year eiio 

2 - completed ) to 12 mnnthe ana 

3 - ruaiplaCfd within last J aoa. 

4 - currently in pro^rtaa 
9 - eot yet underway 

6 • an loing (concinuout) 



appendix 9 (contM) 



Attachemnt 9: 



Output Map 
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APPENDIX 9 (cont’d) 



Attachment 10 ; Distribution of Responsibilities for Profile Items 



Responsible Person (s) 

Team Leader 

Coordinator for Data Collection 
Team Leader 

Coding Resolution Team 
Team Leader 

Team Leader 

T^;:am Leader 
Team Leader 

Coordinator for Data Collection 

Each Interviewer 

Coding Resoluv':icn Team 
Coding Resolution Team 

y • 

Typist, Coordinator of Data Inputs 

Team Leader 
Team Leader 

Coordinator for Data Collection 



Profile Item 

Final Output Map 

Approved Output ^^ap 

Output Index and Coding 
(Form 05) 

Approved Output Labels and 
Codes (Form 05) 

Annotations for Outputs Avail*- 
able to Others (e.g, , published 
documents, instructional packages) 

Complete Bibliographic Entries 
and Ordering Information 
(Form 01 and Worksheet 08) 

Annotated Listing of Project Out- 
puts (by major types or groups) 

Project Roster and Related Out- 
puts for Interview 

Approved Project Roster Descrip- 
tions, Annotated Listings of Pro- 
ject Outputs, Annotations of 
Available Products 

Recap Statements (Forms 06, 07, 

08) 

Recap Coding (Forms 06, 07, 08) 

Review and Approval of Statements 

Collation of Output Catalogue 
Components (Forms 06, 07, 08) 

Contextual Map 

Organizational Structure Figure 

Approval of Contextual Map and 
Organizational Structure 
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Responsiole Person (s) 


Profile Item 


Typist, Coordinator of Dat.a Inputs 


Col la ti or cf Contextual Cues 
(Worksheet 11) 


Typist, Coordinator of Data Inputs 


Collation of Training Snggu a** loii.s 
(Form J2 and Worksheet 13) 


Data Analysis Team 


Hand tally of Inventory Items 
(Form 02) 


Typist, Coordinator of Data Inputs 


Table Summarizing Form 03 Responses 


Profile Writer 


Table Summarizing Interviewee 
Backgrounds 


Each Interviewer 


Inter\'iew Notes on Interrelation- 
ships (Worksheet 14) 


Profile Writer 


Abstract of Project (edit#^d) 


Data Analysis Programmer 


Computer Printouts of Usage of 
Category Sets 


Typist, Coordinator of Data Inputs 


Tables of Usage of Category Sets 


Data Analysis Programmer 


Computer Data on Relation of 
Category Sets of Outputs 


Typist, Coordinator of Data Inputs 


Tables of Category Set Relations 
to Outputs 


Data Analysis Programmer 


Computer Printout of activity 
Significance Means and Percents 
(Form 04) 


Typist, Coordinator of Data Inputs 


Table Summarizing Activity Signif- 
icance 


Profile Writer 


Editing Proposal 


Coordinator for Site Selection 


Summary of Project Selection 
Characteristics 
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APPENDIX 9 (cont'd) 



Worksheet 20 



Profile* Transr.ittal Sheet (Team) 

TO: DATE: 

RE: FROM: 

Attached is a copy of the first rough draft of case profile 

// . Please review it caretully, writing your comments directly 

on the draft and/or on a separate sheet. When reviewing this case 
profile please be particularly sensitive to the following: 

1. Accuracy of content and interpretations . 

2. Completeness of obtained site inf ornation and product 
descriptions . 

Please return the profile to me by , with 

your written comments. 



O 

ERIC 
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Attachment 12 



APPENDIX 9 (cont'd) 



Worksheet 21 



Profile Transmittal Sheet (Coordinators) 



TO: 



DATE: 



RE: 



FROM : 



Attached Is a copy of first rough draft of case profile it 



Please look It over carefully, writing your comments directly on 



I 



the draft and/or on a separate sheet, \\lien reviewing this case 
profile please be particularly sensitive to the follov/ing: 

1* Communicability of profile description for general audi- 
ences, Including readability and clarity of style. 

2. Adequate accomplishment of the conceptual intent of case 
profiles by RDD&E project. 

3. Adequacy of specific information provided for technical 



audiences and potential use in the future of others. 

4. Adequate handling of conf identiali tv of the informatio n 
source . 

5. Adequa handling of the sensitivity of project infor - 
mation and manner of description . 

Please return the profile to me by , with your 
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written comments. 
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Attachment 13 



Worksheet 22 



Profile Transmittal Sheet (Editor) 

TO : DATE : 

RE : FROM : 

Attached is a copy of the site case profile // . Please 

look It over carefully, writing your comments directly on the draft 
and/or on a separate sheet. When reviewing this case profile please 
be particularly sensitive to the following: 



1. 


Completeness of content. 




2. 


Foraat consistency. 




3. 


Readability and clarity of style. 






Provision of definitions of technical terms. 


improper use 




of jargon. 




5. 


Proper spelling, grammar, and syntax. 




6. 


Reasonable compatibility of format and style 


with the other 




profiles. 




7. 


Adequate handling of confidentiality and sensitivity. 



Please return the profile to me by , with 

your written comments. 





. /\ 
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APPtiNDIX 9 (cont'd) 



Worksheet 23 



Profile Transmittal Sheet (Project Director) 



Dear 



Enclosed is a draft copy of the case profile we have prepared for 
your project. To reiterate, the intent of this case profile is to provide 
In-depth information about selected outputs which comprise your 
project. We have not attempted to present a historically complete 
review of your project, but instead have only the snap-shot which time 
allowed. 

We would appreciate your review of the profile in terms of content 
accuracy, interpretations, and editing of project documents contained 
within the profile. Comment may be provided directly on the profile 
and/or on a separate sheet. If you have any concerns regarding the 
profile, please feel free to call collect at 000-000-0000 for: 



please sign and return. Also, please complete and return the attached 
rating sheet. A self-addressed envelope has been provided for the return 
of these forms and any other written comments. 



Dr. John Doe, Project Director 
Dr. Fred Smith, Project Coordinator 
Mr. Gar Smith, Project Coordinator 



by 



Because of the press of time, we need to receive word from you 
• Enclosed is a release form which we ask that you 
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Thank you very much for your cooperation and assistance in this 
matter and tor your participation in the data colJection process. You 
might be interested to know, also, that most of the information has 
been coded for computer storage--avai lable for a variety of sumraativc* 
analyses across many projects. 

Thank you again for your assistance. 




Sincerely , 



John Doe 

Research Professor 



} 



jj 

Enclosure 
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Attachment 15 ; Profile Rating Sheet 

(Case Profile Number 



(PROJECT TITLE] 



To Director (and/or rater) : Following your review of the Case Profile prepared on 

the basis of information obtained from you and your staff, a rating of the 
representativeness of the Profile is requested. To assist in this rating, 
look at each item below and rate it on the basis of the key provided. Items 
(1), (2), and (3) are referenced throughout the profile, with primary concen-- 
tration in Chapters I and II. Items (A), (5), and (6) indicate the location 
in the Profile of the specific items involved. Item (7) is a measure of the 
degree to which the Profile as a V'hole represents the project. 

Rating Schedule 

Check box A, B, or C as follows: 

A = Representative of a majority of operational concerns. 

B = Representative of only a part of operational concerns (concerns 
of significant proportions omitted.) 

C = Major concerns not covered. 





Focus of rating 


A 


B 


c 


Comment , if any 




1. 


Description of organizational 


□ 


n n 




w 




structure 










2. 


Description of organizational 


□ 


□Til 








operations , interrelationships 










3. 


Description of the ends being 


□ 


□ 


□ 








sought by the project 








4, 


Representativeness of all 


n 


□ 


□ 








outputs Indexed (Ch. II) 








5. 


Representativeness of the 


□ 


□ 


□ 








outputs analyzed (Ch. Ill) 








6. 


Representative accuracy of the 


□ 


□ 


□ 








data on outputs (Appendix) 








7. 


Overall representativeness 


□ 


a 


□ 







of the Profile 
General Comment, if any: 




r* 



f 

i 
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^ APPENDIX 9 (conc'd) 
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[ Alcachment 16 : Profile Release Sheet 

I 



Pernission is hereby granted for the inclusion of: 



A CASE PROFILE OF 

A PROJECT TITLED: The Evaluation of the Early 

Childhood Education Program 



A PROJECT OF: Appalachia Educational 

Laboratory, Inc. 

P. 0. Bo:< 1348 

Charleston, West Virginia 25325 



AS PREPARED BY: Teaching Research Division 

Oregon State System of Higher Education 
Monmouth, Oregon 97361 

within the outputs of a project for "The Generation of 
Information to Support Long-Term Manpov:er Studies of and 
Planning for Training Programs in Educational Research, 
Development, Diffusion, and Evaluatio.a" v/hich is being 
carried out under U.S. Office of Education Grant No. 

0-70-4977. 



Project Director: 



Date 
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A GUIDE TO THE OREGON STUDIES IN EDUCATIONAL RDD&E 



Volume I 

SUMMARY REPORT 

An introduction to and overview of the Oregon Studies as a 
whole. The volume contains an outline of the history of the 
Studies, the rationale around which they were designed, the 
context within which they were carried out, and the 
procedures followed in their execution. It also contains a 
description of the projects selected for study, the rationale 
underlying their selection, the criteria and procedures used 
in their selection, and an overview of the data collected on 
each project. Finally, the volume contains an introduction 
to the *'case profiles” that house the data collected on each 
project, the results of all cross-project analyses, and the 
summary recommendations that have been made relative to 
training and the continued study of educational RDD&E 
activities. A brief description of the case study method- 
ology developed within the Studies, an overview of a 
process whereby investigators may query computer-stored 
data files and original interview statements to obtain 
information bearing upon specific questions relating to 
training, manpower, policy, and work performance, and 
supporting data accompany the volume. 



Volume II 

THE LITERATURE OF EDUCATIONAL ROD&E 

A compendium of existing literature that defines, describes, 
differentiates, or relates the activities labeled educational 
research, development, diffusion, evaluation, and various 
combinations thereof. The articles within the volume are 
introduced as a collection. Linking passages provide an 
interpretive context both for individual articles and for the 
sets into which they have been grouped. 



Volume III 

CONCEPTUAL FRAMEWORKS FOR VIEWING 
EDUCATIOMAL RDD&E 

A collection of papers which provide the conceptual 
underpinnings to the Oregon Studies. It contains three 
papers commissioned by the Studies as a basis for con- 
ceptual development, and a paper by staff from Teaching 
Research that describes the conceptual frame that guided 
;and grew with the empirical thrust of the Studies. Each of 



the papers is a major document which defines, differ- 
entiates, and relates one or more facets of educational 
RDD&E and provides a supporting rationale for the 
position adopted. Each paper is accompanied by a formal 
critique, and the set of papers is accompanied by an 
introductory and summary critique. 

Volume IV 

PROFILES OF EXEMPLARY PROJECTS 
IN EDUCATIONAL RDVSlE 

A collection of twenty case profiles that form the data 
base in the Oregon Studies. Printed in three^jrarts, the 
profiles describe five research projects, sevef^eWj 0 pment 
projects, thrbe evaluation projects, and nve||M 
projects. Each profile contains descriptions pf the^h^tiire 
and function of the project being analyzed, tlgp^cific 
outputs expected to emerge from it, the bjierations 
required to produce each output, and the knowledges, 
skills, and sensitivities judged to be essential to . the 
performance of those operations. In addition, each profile 
contains sections dealing with the "^dynamics” of project 
operations and implications that derive from the project for 
preservice staff training. The projects described range from 
small, two-man efforts within university settings to very 
large school district *^rojects” employing several dozen 
staff members. Eighteen of the twenty projects described 
were judged to be illustrative of the kinds of RDD&E 
activities likely to occur within the context of education in 
the future. The twenty projects account for analyses 
around 298 project outputs and interviews with 134 
professional staff members. 

Volume V 

A METHODOLOGY FOR THE STUDY 
OF EDUCATIONAL RDD&E 

A detailed description of the most refined form of the data 
collection methodology developed within the Studies, 
directions to guide its use, and the decision rules needed for 
the volume to function as a users manual. The volume 
includes information on procedures used in site contact, 
site preparation, data reduction and analysis, and profile 
preparation. It also includes information on the category 
sets used in data reduction and the computerized data files 
that contain or provide access to all data collected in the 
Studies. 



Copies of any or all of these volumes 
may be obtained at cost from 
Teaching Research 
Monmouth, Oregon 97361 



